Note: This is test shard 2 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/1
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 02:09:05.942320 27092 test_util.cc:268] Using random seed: 877523750
2024-05-03T02:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:09:05Z Disabled control of system clock
I20240503 02:09:05.952368 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--webserver_interface=127.26.117.62
--webserver_port=0
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20240503 02:09:06.022621 27100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.022753 27100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.022770 27100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.023926 27100 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:06.023957 27100 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:06.023968 27100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:06.023979 27100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.023993 27100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:06.024003 27100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:06.025401 27100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.117.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.025549 27100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.025717 27100 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:06.029428 27105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.029793 27106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.030104 27108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.030614 27100 server_base.cc:1034] running on GCE node
I20240503 02:09:06.030781 27100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.030948 27100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.032071 27100 hybrid_clock.cc:648] HybridClock initialized: now 1714702146032056 us; error 32 us; skew 500 ppm
I20240503 02:09:06.036116 27100 webserver.cc:460] Webserver started at http://127.26.117.62:40493/ using document root <none> and password file <none>
I20240503 02:09:06.036262 27100 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.036296 27100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.036386 27100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.037125 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/instance:
uuid: "c324c9220fd14d3c817b18bbd3774439"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.037401 27100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal/instance:
uuid: "c324c9220fd14d3c817b18bbd3774439"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.038359 27100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:06.038873 27114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.039005 27100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:06.039070 27100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
uuid: "c324c9220fd14d3c817b18bbd3774439"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.039121 27100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.055392 27100 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.055565 27100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.055653 27100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.058705 27100 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.62:37529
I20240503 02:09:06.058732 27166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.62:37529 every 8 connection(s)
I20240503 02:09:06.058948 27100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
I20240503 02:09:06.059427 27167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:06.060767 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.064651 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
W20240503 02:09:06.064953 27117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.117.61:37847: connect: Connection refused (error 111)
W20240503 02:09:06.065358 27167 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.61:37847: Network error: Client connection negotiation failed: client connection to 127.26.117.61:37847: connect: Connection refused (error 111)
I20240503 02:09:06.065434 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27100
I20240503 02:09:06.065515 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal/instance
I20240503 02:09:06.066140 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.61:37847
--webserver_interface=127.26.117.61
--webserver_port=0
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20240503 02:09:06.114523 27167 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } attempt: 1
W20240503 02:09:06.114992 27167 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.61:37847: Network error: Client connection negotiation failed: client connection to 127.26.117.61:37847: connect: Connection refused (error 111)
W20240503 02:09:06.136274 27172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.136402 27172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.136426 27172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.137629 27172 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:06.137662 27172 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:06.137679 27172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:06.137694 27172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.137709 27172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:06.137728 27172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:06.139066 27172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.61:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.117.61
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.139245 27172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.139420 27172 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:06.142969 27178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.143290 27180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.143671 27177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.144049 27172 server_base.cc:1034] running on GCE node
I20240503 02:09:06.144223 27172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.144402 27172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.145509 27172 hybrid_clock.cc:648] HybridClock initialized: now 1714702146145498 us; error 22 us; skew 500 ppm
I20240503 02:09:06.146442 27172 webserver.cc:460] Webserver started at http://127.26.117.61:43671/ using document root <none> and password file <none>
I20240503 02:09:06.146598 27172 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.146636 27172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.146724 27172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.147451 27172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data/instance:
uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.147713 27172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal/instance:
uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.148656 27172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20240503 02:09:06.149175 27186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.149302 27172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.149389 27172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal
uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.149446 27172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.183121 27167 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } attempt: 2
W20240503 02:09:06.183611 27167 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.61:37847: Network error: Client connection negotiation failed: client connection to 127.26.117.61:37847: connect: Connection refused (error 111)
I20240503 02:09:06.187680 27172 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.187876 27172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.187968 27172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.191044 27172 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.61:37847
I20240503 02:09:06.191069 27238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.61:37847 every 8 connection(s)
I20240503 02:09:06.191298 27172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/data/info.pb
I20240503 02:09:06.191779 27239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:06.193153 27239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.196624 27239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.199294 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27172
I20240503 02:09:06.199371 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-1/wal/instance
I20240503 02:09:06.199424 27239 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.199682 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--webserver_interface=127.26.117.60
--webserver_port=0
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20240503 02:09:06.199721 27190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
W20240503 02:09:06.200090 27239 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.60:36369: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:06.249248 27239 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } attempt: 1
W20240503 02:09:06.249799 27239 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.60:36369: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
W20240503 02:09:06.271481 27244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.271627 27244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.271651 27244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.272884 27244 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:06.272922 27244 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:06.272939 27244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:06.272953 27244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.272969 27244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:06.272989 27244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:06.274370 27244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.117.60
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.27244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.274544 27244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.274726 27244 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:09:06.274744 27167 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } attempt: 3
I20240503 02:09:06.277838 27167 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
W20240503 02:09:06.278445 27167 consensus_peers.cc:644] Error getting permanent uuid from config peer 127.26.117.60:36369: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
W20240503 02:09:06.278990 27252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.279369 27244 server_base.cc:1034] running on GCE node
W20240503 02:09:06.280213 27250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.280556 27249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.280762 27244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.280972 27244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.282092 27244 hybrid_clock.cc:648] HybridClock initialized: now 1714702146282077 us; error 26 us; skew 500 ppm
I20240503 02:09:06.283093 27244 webserver.cc:460] Webserver started at http://127.26.117.60:44717/ using document root <none> and password file <none>
I20240503 02:09:06.283267 27244 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.283310 27244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.283406 27244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.284319 27244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/instance:
uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.284580 27244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal/instance:
uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.285578 27244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20240503 02:09:06.286110 27258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.286248 27244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:06.286314 27244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.286365 27244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.292682 27244 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.292870 27244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.292959 27244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.296141 27244 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.60:36369
I20240503 02:09:06.296168 27310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.60:36369 every 8 connection(s)
I20240503 02:09:06.296394 27244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
I20240503 02:09:06.296875 27311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:06.298348 27311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.301687 27311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.302699 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27244
I20240503 02:09:06.302769 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal/instance
I20240503 02:09:06.304270 27311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } has no permanent_uuid. Determining permanent_uuid...
I20240503 02:09:06.307943 27311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap starting.
I20240503 02:09:06.308523 27311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:06.308758 27311 log.cc:824] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:06.309235 27311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: No bootstrap required, opened a new log
I20240503 02:09:06.309592 27167 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } attempt: 1
I20240503 02:09:06.311852 27311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.311996 27311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:06.312038 27311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c75a5f0d0824fafbc14d2a2c1613db1, State: Initialized, Role: FOLLOWER
I20240503 02:09:06.312127 27311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.312373 27311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:06.312350 27318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.312750 27318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.314440 27167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap starting.
I20240503 02:09:06.314879 27167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:06.315086 27167 log.cc:824] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Log is configured to *not* fsync() on all Append() calls
W20240503 02:09:06.314116 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.315481 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.315624 27167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: No bootstrap required, opened a new log
I20240503 02:09:06.316923 27167 raft_consensus.cc:348] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.317013 27167 raft_consensus.cc:374] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:06.317034 27167 raft_consensus.cc:729] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c324c9220fd14d3c817b18bbd3774439, State: Initialized, Role: FOLLOWER
I20240503 02:09:06.317121 27167 consensus_queue.cc:260] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.317375 27167 sys_catalog.cc:564] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:06.317338 27332 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.317508 27332 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.315871 27244 master_runner.cc:186] Error getting master registration for 127.26.117.62:37529: OK, instance_id { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" instance_seqno: 1714702146057859 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20240503 02:09:06.317768 27244 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20240503 02:09:06.318030 27239 consensus_peers.cc:654] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } attempt: 2
W20240503 02:09:06.319850 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.319949 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.322201 27239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Bootstrap starting.
I20240503 02:09:06.322736 27239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:06.320945 27100 master_runner.cc:186] Error getting master registration for 127.26.117.61:37847: OK, instance_id { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" instance_seqno: 1714702146190226 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20240503 02:09:06.322832 27100 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20240503 02:09:06.322957 27239 log.cc:824] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:06.323416 27239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: No bootstrap required, opened a new log
I20240503 02:09:06.324816 27239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.324894 27239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:06.324914 27239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e12d823f5eb49f38d5fdc7ee2185bb4, State: Initialized, Role: FOLLOWER
I20240503 02:09:06.325009 27239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.325227 27239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:06.325438 27346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
W20240503 02:09:06.325685 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.325724 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.326556 27346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: FOLLOWER
W20240503 02:09:06.327142 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.327230 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.328581 27172 master_runner.cc:263] No leader master found from master 9e12d823f5eb49f38d5fdc7ee2185bb4
I20240503 02:09:06.328622 27172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20240503 02:09:06.330186 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.330232 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.335938 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.335992 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.337464 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.337545 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.340417 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.340459 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.346199 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.346239 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.347954 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.348045 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.350666 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.350709 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.356422 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.356465 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.358274 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.358359 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.360896 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.360940 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.366657 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.366699 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.368579 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.368660 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.371133 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.371177 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.376875 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.376917 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.378909 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.379011 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.381389 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.381433 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.389853 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.389896 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.390101 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.390190 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.394925 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.394979 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.400089 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.400132 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.400404 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.400440 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.405157 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.405202 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.410336 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.410380 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.410615 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.410657 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.415412 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.415458 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.420593 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.420639 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.420855 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.420897 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.425623 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.425669 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.430840 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.430886 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.431084 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.431126 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.435863 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.435910 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.438411 27192 tablet.cc:2369] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240503 02:09:06.441088 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.441131 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.441339 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.441375 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.446092 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.446138 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.451288 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.451335 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.451557 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.451599 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.455274 27318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:06.455346 27318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.455552 27318 leader_election.cc:288] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847)
I20240503 02:09:06.455844 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" is_pre_election: true
I20240503 02:09:06.455950 27213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c75a5f0d0824fafbc14d2a2c1613db1 in term 0.
I20240503 02:09:06.456157 27261 leader_election.cc:302] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1, 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters:
I20240503 02:09:06.456277 27318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:06.456324 27318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:06.456343 27318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:06.457130 27318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.457243 27318 leader_election.cc:288] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 1 election: Requested vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847)
I20240503 02:09:06.457489 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4"
I20240503 02:09:06.457549 27213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:06.458277 27213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c75a5f0d0824fafbc14d2a2c1613db1 in term 1.
I20240503 02:09:06.458422 27261 leader_election.cc:302] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1, 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters:
I20240503 02:09:06.458941 27141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c324c9220fd14d3c817b18bbd3774439" is_pre_election: true
I20240503 02:09:06.459069 27141 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c75a5f0d0824fafbc14d2a2c1613db1 in term 0.
I20240503 02:09:06.459177 27140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c324c9220fd14d3c817b18bbd3774439"
I20240503 02:09:06.459224 27140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:06.459897 27140 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c75a5f0d0824fafbc14d2a2c1613db1 in term 1.
I20240503 02:09:06.460858 27318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:06.461014 27318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 LEADER]: Becoming Leader. State: Replica: 9c75a5f0d0824fafbc14d2a2c1613db1, State: Running, Role: LEADER
I20240503 02:09:06.461084 27318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:06.461405 27318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c75a5f0d0824fafbc14d2a2c1613db1. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.461481 27318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: LEADER
W20240503 02:09:06.461791 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.461831 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.461939 27329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.461982 27329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.462122 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.462159 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.462335 27361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:09:06.462493 27361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:09:06.463999 27213 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 1 FOLLOWER]: Refusing update from remote peer 9c75a5f0d0824fafbc14d2a2c1613db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:06.464285 27359 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:06.465185 27346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c75a5f0d0824fafbc14d2a2c1613db1. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.465250 27346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.465616 27140 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Refusing update from remote peer 9c75a5f0d0824fafbc14d2a2c1613db1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:06.465847 27359 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:06.466778 27332 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c75a5f0d0824fafbc14d2a2c1613db1. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.466847 27332 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.467783 27359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.467851 27359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:06.468314 27346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.468386 27346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.468600 27362 mvcc.cc:205] Tried to move back new op lower bound from 7023419991914774528 to 7023419991905062912. Current Snapshot: MvccSnapshot[applied={T|T < 7023419991914774528}]
I20240503 02:09:06.468708 27363 mvcc.cc:205] Tried to move back new op lower bound from 7023419991914774528 to 7023419991905062912. Current Snapshot: MvccSnapshot[applied={T|T < 7023419991914774528}]
I20240503 02:09:06.469199 27361 catalog_manager.cc:1347] Generated new cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:06.469238 27361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20240503 02:09:06.472324 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.472362 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.472491 27357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.472523 27357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.482529 27343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240503 02:09:06.482573 27343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.482787 27357 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:06.482823 27357 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: loading cluster ID for follower catalog manager: success
W20240503 02:09:06.482875 27357 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20240503 02:09:06.482893 27357 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.483711 27318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.483783 27318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:06.484109 27332 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:06.484169 27332 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:06.484347 27365 mvcc.cc:205] Tried to move back new op lower bound from 7023419991914774528 to 7023419991905062912. Current Snapshot: MvccSnapshot[applied={T|T < 7023419991914774528}]
I20240503 02:09:06.492810 27343 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:06.492849 27343 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: success
W20240503 02:09:06.492895 27343 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20240503 02:09:06.492913 27343 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.493099 27357 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20240503 02:09:06.493135 27357 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.503096 27343 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20240503 02:09:06.503131 27343 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20240503 02:09:06.503279 27357 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20240503 02:09:06.503311 27357 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.510682 27361 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:09:06.511175 27361 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:09:06.514508 27343 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: acquiring CA information for follower catalog manager: success
W20240503 02:09:06.514595 27343 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.515779 27357 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: acquiring CA information for follower catalog manager: success
W20240503 02:09:06.515851 27357 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20240503 02:09:06.523586 27361 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Generated new TSK 0
I20240503 02:09:06.523751 27361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:09:06.524906 27343 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:06.526198 27357 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:06.530203 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.117.1:0
--local_ip_for_outbound_sockets=127.26.117.1
--webserver_interface=127.26.117.1
--webserver_port=0
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20240503 02:09:06.603670 27371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.603806 27371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.603824 27371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.605039 27371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.605086 27371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.117.1
I20240503 02:09:06.606477 27371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.117.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.117.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--never_fsync=true
--heap_profile_path=/tmp/kudu.27371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.117.1
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.606652 27371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.606842 27371 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:06.610783 27376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.611060 27379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.611457 27377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.611805 27371 server_base.cc:1034] running on GCE node
I20240503 02:09:06.611935 27371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.612143 27371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.613265 27371 hybrid_clock.cc:648] HybridClock initialized: now 1714702146613246 us; error 29 us; skew 500 ppm
I20240503 02:09:06.614305 27371 webserver.cc:460] Webserver started at http://127.26.117.1:46809/ using document root <none> and password file <none>
I20240503 02:09:06.614473 27371 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.614517 27371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.614615 27371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.615368 27371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data/instance:
uuid: "bd4b79bd8b57443faa6fa5e464570188"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.615624 27371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/instance:
uuid: "bd4b79bd8b57443faa6fa5e464570188"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.616621 27371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20240503 02:09:06.617177 27385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.617341 27371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:06.617419 27371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal
uuid: "bd4b79bd8b57443faa6fa5e464570188"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.617463 27371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.626513 27371 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.626649 27371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.626716 27371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.626843 27371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:09:06.627050 27371 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:09:06.627086 27371 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.627115 27371 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:09:06.627135 27371 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.631750 27371 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.1:46597
I20240503 02:09:06.631781 27498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.1:46597 every 8 connection(s)
I20240503 02:09:06.631989 27371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/data/info.pb
I20240503 02:09:06.633589 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27371
I20240503 02:09:06.633684 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/instance
I20240503 02:09:06.635210 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.117.2:0
--local_ip_for_outbound_sockets=127.26.117.2
--webserver_interface=127.26.117.2
--webserver_port=0
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20240503 02:09:06.636838 27500 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:06.636926 27500 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.636974 27501 heartbeater.cc:347] Connected to a master server at 127.26.117.61:37847
I20240503 02:09:06.636967 27499 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:06.637048 27501 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.637084 27499 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.637104 27500 heartbeater.cc:511] Master 127.26.117.62:37529 requested a full tablet report, sending...
I20240503 02:09:06.637157 27499 heartbeater.cc:511] Master 127.26.117.60:36369 requested a full tablet report, sending...
I20240503 02:09:06.637163 27501 heartbeater.cc:511] Master 127.26.117.61:37847 requested a full tablet report, sending...
I20240503 02:09:06.637516 27275 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:06.637542 27131 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:06.637590 27203 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:06.638178 27275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.117.1:56825
W20240503 02:09:06.709371 27506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.709504 27506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.709522 27506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.710695 27506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.710738 27506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.117.2
I20240503 02:09:06.712038 27506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.117.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.117.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--never_fsync=true
--heap_profile_path=/tmp/kudu.27506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.117.2
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.712179 27506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.712414 27506 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:06.716300 27511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.716542 27514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.716912 27512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.717309 27506 server_base.cc:1034] running on GCE node
I20240503 02:09:06.717489 27506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.717664 27506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.718784 27506 hybrid_clock.cc:648] HybridClock initialized: now 1714702146718772 us; error 22 us; skew 500 ppm
I20240503 02:09:06.719827 27506 webserver.cc:460] Webserver started at http://127.26.117.2:38551/ using document root <none> and password file <none>
I20240503 02:09:06.719990 27506 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.720026 27506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.720096 27506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.720826 27506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data/instance:
uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.721077 27506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/instance:
uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.722069 27506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20240503 02:09:06.722591 27520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.722725 27506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:06.722784 27506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal
uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.722838 27506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.745870 27506 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.746023 27506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.746105 27506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.746259 27506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:09:06.746488 27506 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:09:06.746516 27506 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.746542 27506 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:09:06.746564 27506 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.751243 27506 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.2:38407
I20240503 02:09:06.751273 27633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.2:38407 every 8 connection(s)
I20240503 02:09:06.751490 27506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/data/info.pb
I20240503 02:09:06.755798 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:06.755884 27635 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.756057 27635 heartbeater.cc:511] Master 127.26.117.62:37529 requested a full tablet report, sending...
I20240503 02:09:06.756315 27636 heartbeater.cc:347] Connected to a master server at 127.26.117.61:37847
I20240503 02:09:06.756363 27636 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.756382 27634 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:06.756389 27131 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:06.756438 27634 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.756443 27636 heartbeater.cc:511] Master 127.26.117.61:37847 requested a full tablet report, sending...
I20240503 02:09:06.756552 27634 heartbeater.cc:511] Master 127.26.117.60:36369 requested a full tablet report, sending...
I20240503 02:09:06.756754 27203 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:06.756783 27275 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:06.757148 27275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.117.2:36567
I20240503 02:09:06.759742 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27506
I20240503 02:09:06.759821 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/instance
I20240503 02:09:06.760735 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.117.3:0
--local_ip_for_outbound_sockets=127.26.117.3
--webserver_interface=127.26.117.3
--webserver_port=0
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20240503 02:09:06.834107 27641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:06.834232 27641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:06.834250 27641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:06.835412 27641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:06.835455 27641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.117.3
I20240503 02:09:06.836776 27641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.117.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.117.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--never_fsync=true
--heap_profile_path=/tmp/kudu.27641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.117.3
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:06.836915 27641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:06.837074 27641 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:06.840914 27647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.841153 27649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:06.841495 27646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:06.841871 27641 server_base.cc:1034] running on GCE node
I20240503 02:09:06.842025 27641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:06.842187 27641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:06.843307 27641 hybrid_clock.cc:648] HybridClock initialized: now 1714702146843291 us; error 29 us; skew 500 ppm
I20240503 02:09:06.844308 27641 webserver.cc:460] Webserver started at http://127.26.117.3:45429/ using document root <none> and password file <none>
I20240503 02:09:06.844465 27641 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:06.844502 27641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:06.844591 27641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:09:06.845366 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data/instance:
uuid: "c2be543b8aad4e80a58d6b4ab669faae"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.845623 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/instance:
uuid: "c2be543b8aad4e80a58d6b4ab669faae"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.846563 27641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:06.847096 27655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.847225 27641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:06.847287 27641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal
uuid: "c2be543b8aad4e80a58d6b4ab669faae"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:06.847337 27641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:06.859771 27641 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:06.859921 27641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:06.860013 27641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:06.860158 27641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:09:06.860414 27641 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:09:06.860445 27641 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.860469 27641 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:09:06.860486 27641 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:06.865010 27641 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.3:35961
I20240503 02:09:06.865046 27768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.3:35961 every 8 connection(s)
I20240503 02:09:06.865242 27641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/data/info.pb
I20240503 02:09:06.869570 27771 heartbeater.cc:347] Connected to a master server at 127.26.117.61:37847
I20240503 02:09:06.869650 27771 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.869704 27769 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:06.869752 27770 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:06.869761 27769 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.869804 27770 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:06.869829 27771 heartbeater.cc:511] Master 127.26.117.61:37847 requested a full tablet report, sending...
I20240503 02:09:06.869876 27769 heartbeater.cc:511] Master 127.26.117.60:36369 requested a full tablet report, sending...
I20240503 02:09:06.869931 27770 heartbeater.cc:511] Master 127.26.117.62:37529 requested a full tablet report, sending...
I20240503 02:09:06.870208 27131 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:06.870235 27203 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:06.870254 27275 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:06.870666 27275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.117.3:54265
I20240503 02:09:06.873903 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 27641
I20240503 02:09:06.873965 27092 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/instance
I20240503 02:09:06.875862 27092 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240503 02:09:06.882566 27275 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46956:
name: "default.Table_14ebbad48c4148bfbe83c79ed18aa7a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:06.882607 27273 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46956:
name: "default.Table_85449d38c6fe408bbbaab88ead60af5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:06.882568 27274 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46956:
name: "default.Table_4a2dba8a58cb41e4b633fb85e8712c67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:06.882795 27275 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14ebbad48c4148bfbe83c79ed18aa7a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:06.882829 27274 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a2dba8a58cb41e4b633fb85e8712c67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:06.882891 27273 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85449d38c6fe408bbbaab88ead60af5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:06.883293 27092 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpsd1k2/build/release/bin/kudu with pid 27244
W20240503 02:09:06.888891 27779 connection.cc:535] client connection to 127.26.117.60:36369 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:09:06.889077 27779 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.117.60:36369)
I20240503 02:09:07.023938 27815 raft_consensus.cc:482] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9c75a5f0d0824fafbc14d2a2c1613db1)
I20240503 02:09:07.024013 27815 raft_consensus.cc:504] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:07.024205 27815 leader_election.cc:288] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:07.024341 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c324c9220fd14d3c817b18bbd3774439" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" is_pre_election: true
I20240503 02:09:07.024418 27213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c324c9220fd14d3c817b18bbd3774439 in term 1.
I20240503 02:09:07.024559 27117 leader_election.cc:302] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e12d823f5eb49f38d5fdc7ee2185bb4, c324c9220fd14d3c817b18bbd3774439; no voters:
W20240503 02:09:07.024672 27118 leader_election.cc:334] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369): Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:07.024669 27815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240503 02:09:07.024799 27815 raft_consensus.cc:482] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9c75a5f0d0824fafbc14d2a2c1613db1)
I20240503 02:09:07.024832 27815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:09:07.025537 27815 raft_consensus.cc:504] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:07.025645 27815 leader_election.cc:288] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 election: Requested vote from peers 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:07.025810 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c324c9220fd14d3c817b18bbd3774439" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4"
I20240503 02:09:07.025869 27213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:09:07.026415 27213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c324c9220fd14d3c817b18bbd3774439 in term 2.
I20240503 02:09:07.026556 27117 leader_election.cc:302] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e12d823f5eb49f38d5fdc7ee2185bb4, c324c9220fd14d3c817b18bbd3774439; no voters:
I20240503 02:09:07.026648 27815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:09:07.026742 27815 raft_consensus.cc:686] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 LEADER]: Becoming Leader. State: Replica: c324c9220fd14d3c817b18bbd3774439, State: Running, Role: LEADER
W20240503 02:09:07.026808 27118 leader_election.cc:334] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369): Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:07.026876 27815 consensus_queue.cc:237] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:07.027092 27818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c324c9220fd14d3c817b18bbd3774439. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:07.027168 27818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:07.027307 27819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:09:07.027459 27819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:09:07.027524 27819 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:07.027545 27819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:09:07.027669 27819 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:09:07.027722 27819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Loaded TSK: 0
I20240503 02:09:07.027827 27819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:09:07.031973 27131 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_e917e682d17e4aafa8ac292997819a20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.032053 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_c08e7a3a344c47fca6c55436603da5bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.032104 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_85449d38c6fe408bbbaab88ead60af5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.032176 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_21be8cf71606424e90556df89a7b177d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.032222 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_8c877441540c4c0eaa5fbffb6251791e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.032244 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85449d38c6fe408bbbaab88ead60af5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.032025 27129 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_f2c24c0da8ab4409bf47a9364371f131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.032373 27129 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2c24c0da8ab4409bf47a9364371f131 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.032054 27127 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_4a2dba8a58cb41e4b633fb85e8712c67"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.032080 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_14ebbad48c4148bfbe83c79ed18aa7a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.032280 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c877441540c4c0eaa5fbffb6251791e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032764 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c08e7a3a344c47fca6c55436603da5bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032788 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14ebbad48c4148bfbe83c79ed18aa7a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032244 27131 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e917e682d17e4aafa8ac292997819a20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032245 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21be8cf71606424e90556df89a7b177d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.031983 27130 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_43ec115cd16747a7bbb85a8d5ee9d312"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.031978 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_8fc104db1dd443dfbad7abc8bf4cf259"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.032734 27127 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a2dba8a58cb41e4b633fb85e8712c67 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032981 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fc104db1dd443dfbad7abc8bf4cf259 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.032933 27130 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43ec115cd16747a7bbb85a8d5ee9d312 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.033464 27213 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Refusing update from remote peer c324c9220fd14d3c817b18bbd3774439: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240503 02:09:07.033684 27815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240503 02:09:07.033789 27118 consensus_peers.cc:485] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 -> Peer 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369): Couldn't send request to peer 9c75a5f0d0824fafbc14d2a2c1613db1. Status: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:07.034341 27822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c324c9220fd14d3c817b18bbd3774439. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:07.034425 27822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:07.034941 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:07.035023 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:07.035243 27822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:07.035313 27822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:07.035724 27126 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f2c24c0da8ab4409bf47a9364371f131" } new_table_name: "default.Table_4f2a37398e784453af36b67aca62a69b" modify_external_catalogs: true
I20240503 02:09:07.035964 27131 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_85449d38c6fe408bbbaab88ead60af5f" } new_table_name: "default.Table_d79ff2033c254ab894e36b9a5d9652b4" modify_external_catalogs: true
I20240503 02:09:07.036077 27129 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_14ebbad48c4148bfbe83c79ed18aa7a7" } new_table_name: "default.Table_cfdd82b518844e8eb778d0a3dad992bf" modify_external_catalogs: true
I20240503 02:09:07.036108 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_8c877441540c4c0eaa5fbffb6251791e" } new_table_name: "default.Table_43850e4338eb4b169fb5094bdf03eb07" modify_external_catalogs: true
I20240503 02:09:07.036187 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_e917e682d17e4aafa8ac292997819a20" } new_table_name: "default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2" modify_external_catalogs: true
I20240503 02:09:07.036777 27124 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c08e7a3a344c47fca6c55436603da5bc" } modify_external_catalogs: true
I20240503 02:09:07.036836 27124 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c08e7a3a344c47fca6c55436603da5bc" } modify_external_catalogs: true
W20240503 02:09:07.037303 27126 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a3bf2544755e4642803923e3f713f1e0 consensus state has no leader
I20240503 02:09:07.038141 27123 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4a2dba8a58cb41e4b633fb85e8712c67" } modify_external_catalogs: true
I20240503 02:09:07.038197 27126 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_8fc104db1dd443dfbad7abc8bf4cf259" } modify_external_catalogs: true
I20240503 02:09:07.038259 27126 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_8fc104db1dd443dfbad7abc8bf4cf259" } modify_external_catalogs: true
W20240503 02:09:07.038373 27131 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 432c86ce274e474bb0366ef62dc7012b consensus state has no leader
W20240503 02:09:07.038380 27129 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1b17a061921d4233a6a295749a30c669 has no consensus state
I20240503 02:09:07.038429 27130 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_43ec115cd16747a7bbb85a8d5ee9d312" } modify_external_catalogs: true
I20240503 02:09:07.038465 27130 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_43ec115cd16747a7bbb85a8d5ee9d312" } modify_external_catalogs: true
W20240503 02:09:07.038492 27122 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c517b4922d147aba57e0b6915f7e411 has no consensus state
I20240503 02:09:07.038513 27128 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_21be8cf71606424e90556df89a7b177d" } modify_external_catalogs: true
W20240503 02:09:07.038528 27125 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bbcbc2a3fbdc4dea92027f39c1437f88 has no consensus state
I20240503 02:09:07.038539 27128 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_21be8cf71606424e90556df89a7b177d" } modify_external_catalogs: true
I20240503 02:09:07.038203 27123 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4a2dba8a58cb41e4b633fb85e8712c67" } modify_external_catalogs: true
I20240503 02:09:07.039355 27124 catalog_manager.cc:5766] Not sending DeleteTablet requests; no consensus state for tablet 66841e0b925542ab94686bcd42a9a531
I20240503 02:09:07.040827 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 432c86ce274e474bb0366ef62dc7012b (DEFAULT_TABLE table=default.Table_85449d38c6fe408bbbaab88ead60af5f [id=abfe5a29f62e4f2f884cf0e8ea66f56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.041096 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432c86ce274e474bb0366ef62dc7012b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.040827 27568 tablet_service.cc:1467] Processing CreateTablet for tablet a3bf2544755e4642803923e3f713f1e0 (DEFAULT_TABLE table=default.Table_f2c24c0da8ab4409bf47a9364371f131 [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.041469 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bf2544755e4642803923e3f713f1e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.043365 27837 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.043612 27128 catalog_manager.cc:5766] Not sending DeleteTablet requests; no consensus state for tablet b27ec15fb677445fa10eecce4ed3a346
I20240503 02:09:07.043666 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a3bf2544755e4642803923e3f713f1e0 (DEFAULT_TABLE table=default.Table_f2c24c0da8ab4409bf47a9364371f131 [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.043939 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bf2544755e4642803923e3f713f1e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.044152 27837 tablet_bootstrap.cc:654] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.044412 27837 log.cc:824] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:07.044842 27433 tablet_service.cc:1467] Processing CreateTablet for tablet a3bf2544755e4642803923e3f713f1e0 (DEFAULT_TABLE table=default.Table_f2c24c0da8ab4409bf47a9364371f131 [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.045078 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3bf2544755e4642803923e3f713f1e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.045432 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 432c86ce274e474bb0366ef62dc7012b (DEFAULT_TABLE table=default.Table_85449d38c6fe408bbbaab88ead60af5f [id=abfe5a29f62e4f2f884cf0e8ea66f56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.045511 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432c86ce274e474bb0366ef62dc7012b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.047353 27839 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.047518 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 432c86ce274e474bb0366ef62dc7012b (DEFAULT_TABLE table=default.Table_85449d38c6fe408bbbaab88ead60af5f [id=abfe5a29f62e4f2f884cf0e8ea66f56c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.047598 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 432c86ce274e474bb0366ef62dc7012b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.047833 27839 tablet_bootstrap.cc:654] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.048048 27839 log.cc:824] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:07.048559 27839 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.048610 27839 ts_tablet_manager.cc:1367] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.049449 27837 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.049448 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a (DEFAULT_TABLE table=default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.049510 27837 ts_tablet_manager.cc:1367] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:07.049546 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351554dd21f34b0aa7a77dd2a095ea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.049779 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a (DEFAULT_TABLE table=default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.049856 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351554dd21f34b0aa7a77dd2a095ea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.050000 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a (DEFAULT_TABLE table=default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050089 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351554dd21f34b0aa7a77dd2a095ea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.050562 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f (DEFAULT_TABLE table=default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050626 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 46d81e4231074c598e3751a2e81e13ae (DEFAULT_TABLE table=default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050772 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17a061921d4233a6a295749a30c669 (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050840 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f (DEFAULT_TABLE table=default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050896 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 1c517b4922d147aba57e0b6915f7e411 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.050923 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34058eb194974c92ba28d89e0c7f7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.050997 27563 tablet_service.cc:1467] Processing CreateTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051215 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f (DEFAULT_TABLE table=default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051316 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34058eb194974c92ba28d89e0c7f7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.051237 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 46d81e4231074c598e3751a2e81e13ae (DEFAULT_TABLE table=default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051335 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 1c517b4922d147aba57e0b6915f7e411 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051420 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c517b4922d147aba57e0b6915f7e411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.051505 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d81e4231074c598e3751a2e81e13ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.051846 27431 tablet_service.cc:1467] Processing CreateTablet for tablet 46d81e4231074c598e3751a2e81e13ae (DEFAULT_TABLE table=default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051936 27431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d81e4231074c598e3751a2e81e13ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.052542 27699 tablet_service.cc:1467] Processing CreateTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.052616 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcbc2a3fbdc4dea92027f39c1437f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.052825 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17a061921d4233a6a295749a30c669 (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.052878 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17a061921d4233a6a295749a30c669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.053606 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 1c517b4922d147aba57e0b6915f7e411 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.051295 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17a061921d4233a6a295749a30c669 (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.053660 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c517b4922d147aba57e0b6915f7e411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.053694 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17a061921d4233a6a295749a30c669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.054454 27432 tablet_service.cc:1467] Processing CreateTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.054531 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcbc2a3fbdc4dea92027f39c1437f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.054952 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 351554dd21f34b0aa7a77dd2a095ea1a
I20240503 02:09:07.055150 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 34058eb194974c92ba28d89e0c7f7a3f
I20240503 02:09:07.055470 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.055500 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.055519 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.055650 27561 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.055675 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.055739 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34058eb194974c92ba28d89e0c7f7a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.055724 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.055801 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17a061921d4233a6a295749a30c669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.055891 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcbc2a3fbdc4dea92027f39c1437f88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.055975 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46d81e4231074c598e3751a2e81e13ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.057358 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c517b4922d147aba57e0b6915f7e411. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.057474 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 46d81e4231074c598e3751a2e81e13ae
I20240503 02:09:07.057703 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.057907 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
W20240503 02:09:07.058107 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
W20240503 02:09:07.058180 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
I20240503 02:09:07.058351 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.058306 27839 raft_consensus.cc:348] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.058416 27839 raft_consensus.cc:374] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.058440 27839 raft_consensus.cc:729] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.058526 27839 consensus_queue.cc:260] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.058740 27839 ts_tablet_manager.cc:1398] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20240503 02:09:07.058866 27499 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.059139 27839 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:07.059298 27499 heartbeater.cc:644] Failed to heartbeat to 127.26.117.60:36369 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:07.059672 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.059753 27839 tablet_bootstrap.cc:654] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:07.060210 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
W20240503 02:09:07.060305 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
W20240503 02:09:07.060345 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
W20240503 02:09:07.060537 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
I20240503 02:09:07.060503 27837 raft_consensus.cc:348] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:07.060688 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
I20240503 02:09:07.060700 27837 raft_consensus.cc:374] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.060739 27837 raft_consensus.cc:729] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
W20240503 02:09:07.060868 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
I20240503 02:09:07.060874 27837 consensus_queue.cc:260] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.061097 27837 ts_tablet_manager.cc:1398] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20240503 02:09:07.061136 27634 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.061435 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.061648 27635 heartbeater.cc:503] Master 127.26.117.62:37529 was elected leader, sending a full tablet report...
I20240503 02:09:07.061720 27837 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:07.061741 27634 heartbeater.cc:644] Failed to heartbeat to 127.26.117.60:36369 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:07.061956 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.062005 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.062024 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.062045 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.062062 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.062248 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.062287 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.062291 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
W20240503 02:09:07.062427 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
W20240503 02:09:07.062492 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
W20240503 02:09:07.062539 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
I20240503 02:09:07.062741 27837 tablet_bootstrap.cc:654] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.063002 27838 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.063380 27838 tablet_bootstrap.cc:654] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.063580 27838 log.cc:824] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:07.063872 27837 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.063911 27837 ts_tablet_manager.cc:1367] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.064030 27837 raft_consensus.cc:348] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.064044 27838 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.064074 27837 raft_consensus.cc:374] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.064088 27838 ts_tablet_manager.cc:1367] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240503 02:09:07.064090 27837 raft_consensus.cc:729] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.064141 27837 consensus_queue.cc:260] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.064221 27837 ts_tablet_manager.cc:1398] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:07.064438 27118 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1c517b4922d147aba57e0b6915f7e411 consensus state has no leader
W20240503 02:09:07.064496 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
I20240503 02:09:07.064673 27837 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.064663 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.065004 27837 tablet_bootstrap.cc:654] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.065459 27500 heartbeater.cc:503] Master 127.26.117.62:37529 was elected leader, sending a full tablet report...
I20240503 02:09:07.065579 27839 tablet_bootstrap.cc:492] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.065616 27839 ts_tablet_manager.cc:1367] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:07.065636 27838 raft_consensus.cc:348] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.065734 27839 raft_consensus.cc:348] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.065752 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.065778 27838 raft_consensus.cc:374] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.065788 27839 raft_consensus.cc:374] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.065790 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.065785 27837 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.065804 27838 raft_consensus.cc:729] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.065809 27839 raft_consensus.cc:729] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.065812 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.065819 27837 ts_tablet_manager.cc:1367] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.065834 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.065860 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.065851 27839 consensus_queue.cc:260] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.065877 27838 consensus_queue.cc:260] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.065960 27837 raft_consensus.cc:348] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.066008 27837 raft_consensus.cc:374] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.066006 27839 ts_tablet_manager.cc:1398] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:07.066025 27837 raft_consensus.cc:729] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.066064 27838 ts_tablet_manager.cc:1398] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240503 02:09:07.066073 27839 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.066367 27838 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.066361 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.066413 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.066417 27769 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.066478 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.066380 27839 tablet_bootstrap.cc:654] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:07.066850 27769 heartbeater.cc:644] Failed to heartbeat to 127.26.117.60:36369 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
I20240503 02:09:07.066937 27838 tablet_bootstrap.cc:654] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.067106 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
W20240503 02:09:07.067106 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
W20240503 02:09:07.067209 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
I20240503 02:09:07.067214 27837 consensus_queue.cc:260] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.067246 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
W20240503 02:09:07.067251 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
I20240503 02:09:07.067912 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.068058 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.068143 27770 heartbeater.cc:503] Master 127.26.117.62:37529 was elected leader, sending a full tablet report...
I20240503 02:09:07.067315 27837 ts_tablet_manager.cc:1398] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240503 02:09:07.068306 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
I20240503 02:09:07.068418 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.068464 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.068485 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.068504 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.068521 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.068746 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.068784 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.068830 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.068861 27844 tablet_replica.cc:330] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.068925 27844 raft_consensus.cc:2229] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
W20240503 02:09:07.068931 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 351554dd21f34b0aa7a77dd2a095ea1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 351554dd21f34b0aa7a77dd2a095ea1a already in progress: creating tablet
I20240503 02:09:07.068995 27844 raft_consensus.cc:2258] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:07.069007 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
I20240503 02:09:07.069099 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.069162 27837 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.069218 27844 ts_tablet_manager.cc:1875] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240503 02:09:07.069252 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 46d81e4231074c598e3751a2e81e13ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46d81e4231074c598e3751a2e81e13ae already in progress: creating tablet
I20240503 02:09:07.069670 27839 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.069706 27839 ts_tablet_manager.cc:1367] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.069765 27838 tablet_bootstrap.cc:492] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.069824 27838 ts_tablet_manager.cc:1367] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.069828 27839 raft_consensus.cc:348] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.069886 27839 raft_consensus.cc:374] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.069905 27839 raft_consensus.cc:729] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.069944 27839 consensus_queue.cc:260] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.069967 27838 raft_consensus.cc:348] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.070015 27839 ts_tablet_manager.cc:1398] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.070021 27838 raft_consensus.cc:374] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.070039 27838 raft_consensus.cc:729] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.070060 27839 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.070087 27838 consensus_queue.cc:260] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.070330 27838 ts_tablet_manager.cc:1398] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.070387 27838 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.070389 27839 tablet_bootstrap.cc:654] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.070703 27838 tablet_bootstrap.cc:654] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.071074 27844 ts_tablet_manager.cc:1888] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.071084 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.071133 27844 log.cc:1196] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/351554dd21f34b0aa7a77dd2a095ea1a
I20240503 02:09:07.071298 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.071314 27844 ts_tablet_manager.cc:1909] T 351554dd21f34b0aa7a77dd2a095ea1a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.071401 27837 tablet_bootstrap.cc:654] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.071615 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 351554dd21f34b0aa7a77dd2a095ea1a (table default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]) successfully deleted
I20240503 02:09:07.071661 27845 tablet_replica.cc:330] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.071732 27845 raft_consensus.cc:2229] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.071796 27845 raft_consensus.cc:2258] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.071885 27838 tablet_bootstrap.cc:492] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.071926 27838 ts_tablet_manager.cc:1367] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.071987 27837 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.072028 27837 ts_tablet_manager.cc:1367] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.072046 27838 raft_consensus.cc:348] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.072106 27838 raft_consensus.cc:374] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.072137 27838 raft_consensus.cc:729] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.072145 27837 raft_consensus.cc:348] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.072198 27837 raft_consensus.cc:374] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.072196 27838 consensus_queue.cc:260] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.072224 27837 raft_consensus.cc:729] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.072288 27838 ts_tablet_manager.cc:1398] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.072366 27838 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.072454 27845 ts_tablet_manager.cc:1875] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.072649 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 351554dd21f34b0aa7a77dd2a095ea1a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.072723 27838 tablet_bootstrap.cc:654] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.072731 27843 tablet_replica.cc:330] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.072852 27843 raft_consensus.cc:2229] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.072907 27843 raft_consensus.cc:2258] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.073256 27838 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.073340 27838 ts_tablet_manager.cc:1367] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.073448 27843 ts_tablet_manager.cc:1875] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.073483 27838 raft_consensus.cc:348] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.073522 27838 raft_consensus.cc:374] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.073535 27838 raft_consensus.cc:729] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.073576 27838 consensus_queue.cc:260] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.073690 27838 ts_tablet_manager.cc:1398] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.073747 27838 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.073997 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.074178 27838 tablet_bootstrap.cc:654] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.074651 27838 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.074688 27838 ts_tablet_manager.cc:1367] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.074795 27838 raft_consensus.cc:348] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.074836 27838 raft_consensus.cc:374] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.074852 27838 raft_consensus.cc:729] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.074898 27838 consensus_queue.cc:260] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.074930 27837 consensus_queue.cc:260] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.075022 27838 ts_tablet_manager.cc:1398] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.075091 27838 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.075345 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.075399 27843 ts_tablet_manager.cc:1888] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.075417 27838 tablet_bootstrap.cc:654] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.075448 27843 log.cc:1196] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/351554dd21f34b0aa7a77dd2a095ea1a
I20240503 02:09:07.075582 27837 ts_tablet_manager.cc:1398] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.075614 27843 ts_tablet_manager.cc:1909] T 351554dd21f34b0aa7a77dd2a095ea1a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.075637 27837 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.075845 27843 tablet_replica.cc:330] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.075879 27843 raft_consensus.cc:2229] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.075901 27843 raft_consensus.cc:2258] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.075906 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 351554dd21f34b0aa7a77dd2a095ea1a (table default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]) successfully deleted
I20240503 02:09:07.076045 27843 ts_tablet_manager.cc:1875] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.076167 27845 ts_tablet_manager.cc:1888] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.076211 27845 log.cc:1196] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/351554dd21f34b0aa7a77dd2a095ea1a
I20240503 02:09:07.076460 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.076503 27837 tablet_bootstrap.cc:654] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.076915 27845 ts_tablet_manager.cc:1909] T 351554dd21f34b0aa7a77dd2a095ea1a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.076974 27837 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.077003 27837 ts_tablet_manager.cc:1367] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.077097 27837 raft_consensus.cc:348] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077167 27843 ts_tablet_manager.cc:1888] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.077183 27837 raft_consensus.cc:374] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.077204 27837 raft_consensus.cc:729] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.077209 27843 log.cc:1196] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/46d81e4231074c598e3751a2e81e13ae
I20240503 02:09:07.077214 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 351554dd21f34b0aa7a77dd2a095ea1a (table default.Table_43ec115cd16747a7bbb85a8d5ee9d312 [id=e0bb5c22710e4a928d3b8765761e1c2d]) successfully deleted
I20240503 02:09:07.077262 27837 consensus_queue.cc:260] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077376 27843 ts_tablet_manager.cc:1909] T 46d81e4231074c598e3751a2e81e13ae P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.077440 27838 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.077474 27838 ts_tablet_manager.cc:1367] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.077533 27839 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.077569 27839 ts_tablet_manager.cc:1367] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:07.077587 27838 raft_consensus.cc:348] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077636 27838 raft_consensus.cc:374] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.077652 27838 raft_consensus.cc:729] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.077682 27839 raft_consensus.cc:348] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077693 27838 consensus_queue.cc:260] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077728 27839 raft_consensus.cc:374] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.077746 27839 raft_consensus.cc:729] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.077781 27838 ts_tablet_manager.cc:1398] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.077790 27839 consensus_queue.cc:260] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.077826 27838 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.077859 27839 ts_tablet_manager.cc:1398] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.077925 27839 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.077970 27837 ts_tablet_manager.cc:1398] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.078140 27838 tablet_bootstrap.cc:654] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.078205 27839 tablet_bootstrap.cc:654] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.078341 27837 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.078712 27837 tablet_bootstrap.cc:654] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.078974 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.079260 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.079314 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.079418 27844 tablet_replica.cc:330] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.079428 27839 tablet_bootstrap.cc:492] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.079473 27839 ts_tablet_manager.cc:1367] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.079483 27844 raft_consensus.cc:2229] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.079517 27844 raft_consensus.cc:2258] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.079591 27839 raft_consensus.cc:348] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.079639 27839 raft_consensus.cc:374] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.079656 27839 raft_consensus.cc:729] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.079666 27837 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.079694 27837 ts_tablet_manager.cc:1367] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.079699 27839 consensus_queue.cc:260] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.079787 27839 ts_tablet_manager.cc:1398] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.079807 27837 raft_consensus.cc:348] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.079857 27839 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.079938 27837 raft_consensus.cc:374] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.079962 27844 ts_tablet_manager.cc:1875] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.079991 27837 raft_consensus.cc:729] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.080071 27837 consensus_queue.cc:260] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.080160 27839 tablet_bootstrap.cc:654] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.080420 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.080662 27837 ts_tablet_manager.cc:1398] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.081135 27844 ts_tablet_manager.cc:1888] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.081146 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.081195 27844 log.cc:1196] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/46d81e4231074c598e3751a2e81e13ae
I20240503 02:09:07.081254 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 46d81e4231074c598e3751a2e81e13ae (table default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]) successfully deleted
I20240503 02:09:07.081341 27843 tablet_replica.cc:330] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.081343 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 46d81e4231074c598e3751a2e81e13ae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.081363 27844 ts_tablet_manager.cc:1909] T 46d81e4231074c598e3751a2e81e13ae P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.081372 27845 tablet_replica.cc:330] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.081378 27843 raft_consensus.cc:2229] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.081405 27843 raft_consensus.cc:2258] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.081409 27845 raft_consensus.cc:2229] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.081434 27845 raft_consensus.cc:2258] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.081532 27845 ts_tablet_manager.cc:1875] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.081125 27837 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.081666 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 46d81e4231074c598e3751a2e81e13ae (table default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]) successfully deleted
I20240503 02:09:07.081830 27843 ts_tablet_manager.cc:1875] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.082052 27837 tablet_bootstrap.cc:654] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.082509 27837 tablet_bootstrap.cc:492] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.082545 27837 ts_tablet_manager.cc:1367] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.082599 27845 ts_tablet_manager.cc:1888] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.082659 27845 log.cc:1196] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/34058eb194974c92ba28d89e0c7f7a3f
I20240503 02:09:07.082649 27837 raft_consensus.cc:348] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.082697 27837 raft_consensus.cc:374] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.082748 27837 raft_consensus.cc:729] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.082803 27845 ts_tablet_manager.cc:1909] T 34058eb194974c92ba28d89e0c7f7a3f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.082808 27837 consensus_queue.cc:260] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.082844 27843 ts_tablet_manager.cc:1888] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.082947 27843 log.cc:1196] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/34058eb194974c92ba28d89e0c7f7a3f
I20240503 02:09:07.083145 27839 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.083184 27839 ts_tablet_manager.cc:1367] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.083307 27839 raft_consensus.cc:348] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.083339 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.083356 27839 raft_consensus.cc:374] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.083375 27839 raft_consensus.cc:729] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.083420 27837 ts_tablet_manager.cc:1398] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.083413 27839 consensus_queue.cc:260] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:07.083492 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 34058eb194974c92ba28d89e0c7f7a3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 34058eb194974c92ba28d89e0c7f7a3f already in progress: creating tablet
I20240503 02:09:07.083504 27839 ts_tablet_manager.cc:1398] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.083549 27839 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.083853 27839 tablet_bootstrap.cc:654] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.083884 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 34058eb194974c92ba28d89e0c7f7a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.083951 27844 tablet_replica.cc:330] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.083992 27844 raft_consensus.cc:2229] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.084048 27844 raft_consensus.cc:2258] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.083989 27837 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.084167 27844 ts_tablet_manager.cc:1875] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.084367 27837 tablet_bootstrap.cc:654] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.084439 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.084736 27839 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.084787 27839 ts_tablet_manager.cc:1367] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.084862 27837 tablet_bootstrap.cc:492] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.084911 27837 ts_tablet_manager.cc:1367] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.084978 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 34058eb194974c92ba28d89e0c7f7a3f (table default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]) successfully deleted
I20240503 02:09:07.085026 27837 raft_consensus.cc:348] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.085088 27837 raft_consensus.cc:374] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.085148 27837 raft_consensus.cc:729] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.085197 27837 consensus_queue.cc:260] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.085254 27839 raft_consensus.cc:348] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.085294 27837 ts_tablet_manager.cc:1398] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.085409 27839 raft_consensus.cc:374] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.085415 27844 ts_tablet_manager.cc:1888] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.085451 27844 log.cc:1196] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/34058eb194974c92ba28d89e0c7f7a3f
I20240503 02:09:07.085374 27845 tablet_replica.cc:330] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.085559 27845 raft_consensus.cc:2229] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.085572 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.085609 27844 ts_tablet_manager.cc:1909] T 34058eb194974c92ba28d89e0c7f7a3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.085630 27845 raft_consensus.cc:2258] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.083199 27838 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.085776 27845 ts_tablet_manager.cc:1875] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.085778 27838 ts_tablet_manager.cc:1367] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:07.085844 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 34058eb194974c92ba28d89e0c7f7a3f (table default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]) successfully deleted
I20240503 02:09:07.085901 27838 raft_consensus.cc:348] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.085952 27838 raft_consensus.cc:374] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.085968 27838 raft_consensus.cc:729] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.086004 27838 consensus_queue.cc:260] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.086082 27838 ts_tablet_manager.cc:1398] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.086125 27838 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.086431 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.086508 27838 tablet_bootstrap.cc:654] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.086758 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.086758 27845 ts_tablet_manager.cc:1888] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.086812 27845 log.cc:1196] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/46d81e4231074c598e3751a2e81e13ae
I20240503 02:09:07.086884 27843 ts_tablet_manager.cc:1909] T 34058eb194974c92ba28d89e0c7f7a3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.086939 27845 ts_tablet_manager.cc:1909] T 46d81e4231074c598e3751a2e81e13ae P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.086958 27838 tablet_bootstrap.cc:492] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.087000 27838 ts_tablet_manager.cc:1367] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.085458 27839 raft_consensus.cc:729] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.087124 27838 raft_consensus.cc:348] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.087142 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 34058eb194974c92ba28d89e0c7f7a3f (table default.Table_8fc104db1dd443dfbad7abc8bf4cf259 [id=40a89f2ca2644efeba9f5885b050d89a]) successfully deleted
I20240503 02:09:07.087164 27838 raft_consensus.cc:374] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.087183 27838 raft_consensus.cc:729] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.087198 27839 consensus_queue.cc:260] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.087224 27838 consensus_queue.cc:260] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.087291 27838 ts_tablet_manager.cc:1398] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.087293 27839 ts_tablet_manager.cc:1398] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.087338 27839 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.087644 27839 tablet_bootstrap.cc:654] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.088119 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 46d81e4231074c598e3751a2e81e13ae (table default.Table_4a2dba8a58cb41e4b633fb85e8712c67 [id=8bcdc3aa90f347c9bb9ca31a2e081b06]) successfully deleted
I20240503 02:09:07.088150 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.088173 27839 tablet_bootstrap.cc:492] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.088202 27839 ts_tablet_manager.cc:1367] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.088227 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.088354 27839 raft_consensus.cc:348] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.088416 27839 raft_consensus.cc:374] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.088443 27839 raft_consensus.cc:729] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.088477 27839 consensus_queue.cc:260] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.088554 27839 ts_tablet_manager.cc:1398] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.088771 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.089198 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.090920 27130 master_service.cc:946] ReplaceTablet: received request to replace tablet a3bf2544755e4642803923e3f713f1e0 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.090893 27126 master_service.cc:946] ReplaceTablet: received request to replace tablet bbcbc2a3fbdc4dea92027f39c1437f88 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.090948 27124 master_service.cc:946] ReplaceTablet: received request to replace tablet 1b17a061921d4233a6a295749a30c669 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.090957 27128 master_service.cc:946] ReplaceTablet: received request to replace tablet 1c517b4922d147aba57e0b6915f7e411 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.090906 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet 1b17a061921d4233a6a295749a30c669 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.091933 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet bbcbc2a3fbdc4dea92027f39c1437f88
I20240503 02:09:07.092018 27126 catalog_manager.cc:6554] ReplaceTablet: tablet bbcbc2a3fbdc4dea92027f39c1437f88 deleted and replaced by tablet fdd36ce35a6546f294c540534534253a
I20240503 02:09:07.092201 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 with delete_type TABLET_DATA_DELETED (replaced by tablet fdd36ce35a6546f294c540534534253a) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.092269 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 with delete_type TABLET_DATA_DELETED (replaced by tablet fdd36ce35a6546f294c540534534253a) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.092283 27843 tablet_replica.cc:330] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.092329 27843 raft_consensus.cc:2229] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.092334 27844 tablet_replica.cc:330] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.092357 27843 raft_consensus.cc:2258] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.092368 27844 raft_consensus.cc:2229] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.092393 27844 raft_consensus.cc:2258] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.092469 27843 ts_tablet_manager.cc:1875] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.092496 27844 ts_tablet_manager.cc:1875] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.092610 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet bbcbc2a3fbdc4dea92027f39c1437f88 with delete_type TABLET_DATA_DELETED (replaced by tablet fdd36ce35a6546f294c540534534253a) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.092697 27845 tablet_replica.cc:330] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.092733 27845 raft_consensus.cc:2229] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.092754 27845 raft_consensus.cc:2258] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.092855 27845 ts_tablet_manager.cc:1875] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.093658 27128 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 1c517b4922d147aba57e0b6915f7e411
I20240503 02:09:07.093686 27845 ts_tablet_manager.cc:1888] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.093704 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 1b17a061921d4233a6a295749a30c669
I20240503 02:09:07.093729 27845 log.cc:1196] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/bbcbc2a3fbdc4dea92027f39c1437f88
I20240503 02:09:07.093748 27128 catalog_manager.cc:6554] ReplaceTablet: tablet 1c517b4922d147aba57e0b6915f7e411 deleted and replaced by tablet 49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.093760 27124 catalog_manager.cc:6554] ReplaceTablet: tablet 1b17a061921d4233a6a295749a30c669 deleted and replaced by tablet ee7d11b2bcc843aba268fe7007fba27d
I20240503 02:09:07.093861 27845 ts_tablet_manager.cc:1909] T bbcbc2a3fbdc4dea92027f39c1437f88 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.093676 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet a3bf2544755e4642803923e3f713f1e0
I20240503 02:09:07.094079 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17a061921d4233a6a295749a30c669 with delete_type TABLET_DATA_DELETED (replaced by tablet ee7d11b2bcc843aba268fe7007fba27d) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.094126 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c517b4922d147aba57e0b6915f7e411 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a31c77c64c4261b5b02dd0acb9d9b0) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.094151 27130 catalog_manager.cc:6554] ReplaceTablet: tablet a3bf2544755e4642803923e3f713f1e0 deleted and replaced by tablet 698f32409eba4b2f9c0b33ef4b76b494
I20240503 02:09:07.094177 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17a061921d4233a6a295749a30c669 with delete_type TABLET_DATA_DELETED (replaced by tablet ee7d11b2bcc843aba268fe7007fba27d) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.094233 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c517b4922d147aba57e0b6915f7e411 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a31c77c64c4261b5b02dd0acb9d9b0) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.094347 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bf2544755e4642803923e3f713f1e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 698f32409eba4b2f9c0b33ef4b76b494) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.094357 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bf2544755e4642803923e3f713f1e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 698f32409eba4b2f9c0b33ef4b76b494) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.094383 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c517b4922d147aba57e0b6915f7e411 with delete_type TABLET_DATA_DELETED (replaced by tablet 49a31c77c64c4261b5b02dd0acb9d9b0) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.094408 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17a061921d4233a6a295749a30c669 with delete_type TABLET_DATA_DELETED (replaced by tablet ee7d11b2bcc843aba268fe7007fba27d) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.094429 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet a3bf2544755e4642803923e3f713f1e0 with delete_type TABLET_DATA_DELETED (replaced by tablet 698f32409eba4b2f9c0b33ef4b76b494) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.094538 27844 ts_tablet_manager.cc:1888] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.094573 27844 log.cc:1196] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/bbcbc2a3fbdc4dea92027f39c1437f88
I20240503 02:09:07.094707 27844 ts_tablet_manager.cc:1909] T bbcbc2a3fbdc4dea92027f39c1437f88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.094828 27843 ts_tablet_manager.cc:1888] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.094864 27843 log.cc:1196] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/bbcbc2a3fbdc4dea92027f39c1437f88
I20240503 02:09:07.094870 27844 tablet_replica.cc:330] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.094902 27844 raft_consensus.cc:2229] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.094925 27844 raft_consensus.cc:2258] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.094998 27843 ts_tablet_manager.cc:1909] T bbcbc2a3fbdc4dea92027f39c1437f88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.095281 27701 tablet_service.cc:1467] Processing CreateTablet for tablet fdd36ce35a6546f294c540534534253a (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.095305 27568 tablet_service.cc:1467] Processing CreateTablet for tablet fdd36ce35a6546f294c540534534253a (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.095302 27844 ts_tablet_manager.cc:1875] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.095366 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd36ce35a6546f294c540534534253a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.095391 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd36ce35a6546f294c540534534253a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.095659 27430 tablet_service.cc:1467] Processing CreateTablet for tablet fdd36ce35a6546f294c540534534253a (DEFAULT_TABLE table=default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.095737 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdd36ce35a6546f294c540534534253a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.096349 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.096499 27838 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.096558 27844 ts_tablet_manager.cc:1888] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.096594 27844 log.cc:1196] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1c517b4922d147aba57e0b6915f7e411
I20240503 02:09:07.096737 27844 ts_tablet_manager.cc:1909] T 1c517b4922d147aba57e0b6915f7e411 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.096729 27839 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.096817 27838 tablet_bootstrap.cc:654] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.096971 27837 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.097064 27839 tablet_bootstrap.cc:654] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.097129 27843 tablet_replica.cc:330] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.097167 27843 raft_consensus.cc:2229] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.097190 27843 raft_consensus.cc:2258] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.097285 27837 tablet_bootstrap.cc:654] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.097440 27843 ts_tablet_manager.cc:1875] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.097460 27844 tablet_replica.cc:330] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.097496 27844 raft_consensus.cc:2229] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.097519 27844 raft_consensus.cc:2258] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.097568 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.097651 27845 tablet_replica.cc:330] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.097692 27845 raft_consensus.cc:2229] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.097715 27845 raft_consensus.cc:2258] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.097813 27564 tablet_service.cc:1467] Processing CreateTablet for tablet ee7d11b2bcc843aba268fe7007fba27d (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.097826 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.097853 27845 ts_tablet_manager.cc:1875] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.097884 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7d11b2bcc843aba268fe7007fba27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.097981 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a31c77c64c4261b5b02dd0acb9d9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.098501 27843 ts_tablet_manager.cc:1888] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.098541 27843 log.cc:1196] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1b17a061921d4233a6a295749a30c669
I20240503 02:09:07.098668 27843 ts_tablet_manager.cc:1909] T 1b17a061921d4233a6a295749a30c669 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.098837 27845 ts_tablet_manager.cc:1888] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.098878 27845 log.cc:1196] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1b17a061921d4233a6a295749a30c669
I20240503 02:09:07.098878 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.098968 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a31c77c64c4261b5b02dd0acb9d9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.098948 27700 tablet_service.cc:1467] Processing CreateTablet for tablet ee7d11b2bcc843aba268fe7007fba27d (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.099011 27845 ts_tablet_manager.cc:1909] T 1b17a061921d4233a6a295749a30c669 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.099014 27843 tablet_replica.cc:330] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.099031 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7d11b2bcc843aba268fe7007fba27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.099048 27843 raft_consensus.cc:2229] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.099074 27843 raft_consensus.cc:2258] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.099184 27843 ts_tablet_manager.cc:1875] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.099381 27845 tablet_replica.cc:330] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.099419 27845 raft_consensus.cc:2229] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.099442 27845 raft_consensus.cc:2258] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.099574 27845 ts_tablet_manager.cc:1875] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.099977 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.100114 27843 ts_tablet_manager.cc:1888] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.100155 27843 log.cc:1196] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1c517b4922d147aba57e0b6915f7e411
I20240503 02:09:07.100188 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.100343 27843 ts_tablet_manager.cc:1909] T 1c517b4922d147aba57e0b6915f7e411 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.100426 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 (DEFAULT_TABLE table=default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.100546 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698f32409eba4b2f9c0b33ef4b76b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.100574 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet bbcbc2a3fbdc4dea92027f39c1437f88 (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.100620 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.100618 27838 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.100656 27838 ts_tablet_manager.cc:1367] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.100760 27432 tablet_service.cc:1467] Processing CreateTablet for tablet ee7d11b2bcc843aba268fe7007fba27d (DEFAULT_TABLE table=default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.100778 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (DEFAULT_TABLE table=default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.100780 27838 raft_consensus.cc:348] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.100831 27838 raft_consensus.cc:374] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.100840 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 (DEFAULT_TABLE table=default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.100857 27838 raft_consensus.cc:729] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.100889 27838 consensus_queue.cc:260] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.100906 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 (DEFAULT_TABLE table=default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.100970 27838 ts_tablet_manager.cc:1398] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.101051 27838 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.101069 27844 ts_tablet_manager.cc:1875] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.101372 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
W20240503 02:09:07.101440 27123 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fdd36ce35a6546f294c540534534253a consensus state has no leader
I20240503 02:09:07.100983 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698f32409eba4b2f9c0b33ef4b76b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.101807 27845 ts_tablet_manager.cc:1888] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.101846 27845 log.cc:1196] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1c517b4922d147aba57e0b6915f7e411
I20240503 02:09:07.101920 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee7d11b2bcc843aba268fe7007fba27d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.101979 27845 ts_tablet_manager.cc:1909] T 1c517b4922d147aba57e0b6915f7e411 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.102082 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a31c77c64c4261b5b02dd0acb9d9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.102104 27844 ts_tablet_manager.cc:1888] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.102137 27844 log.cc:1196] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1b17a061921d4233a6a295749a30c669
I20240503 02:09:07.102173 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 698f32409eba4b2f9c0b33ef4b76b494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.102263 27844 ts_tablet_manager.cc:1909] T 1b17a061921d4233a6a295749a30c669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.102602 27844 tablet_replica.cc:330] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.102640 27844 raft_consensus.cc:2229] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.102667 27844 raft_consensus.cc:2258] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.102777 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1b17a061921d4233a6a295749a30c669 (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.102936 27844 ts_tablet_manager.cc:1875] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.102952 27838 tablet_bootstrap.cc:654] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.103168 27845 tablet_replica.cc:330] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.103209 27845 raft_consensus.cc:2229] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.103233 27845 raft_consensus.cc:2258] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.103220 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.103247 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1c517b4922d147aba57e0b6915f7e411 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.103243 27843 tablet_replica.cc:330] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.103296 27843 raft_consensus.cc:2229] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.103322 27843 raft_consensus.cc:2258] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.103451 27843 ts_tablet_manager.cc:1875] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.103598 27839 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.103646 27839 ts_tablet_manager.cc:1367] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:07.103780 27839 raft_consensus.cc:348] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.103837 27839 raft_consensus.cc:374] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.103859 27839 raft_consensus.cc:729] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.103895 27839 consensus_queue.cc:260] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.104316 27845 ts_tablet_manager.cc:1875] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.104400 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.104650 27843 ts_tablet_manager.cc:1888] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.104699 27843 log.cc:1196] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a3bf2544755e4642803923e3f713f1e0
I20240503 02:09:07.104808 27844 ts_tablet_manager.cc:1888] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.104825 27843 ts_tablet_manager.cc:1909] T a3bf2544755e4642803923e3f713f1e0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.104864 27844 log.cc:1196] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a3bf2544755e4642803923e3f713f1e0
I20240503 02:09:07.105013 27844 ts_tablet_manager.cc:1909] T a3bf2544755e4642803923e3f713f1e0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.105167 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.104328 27839 ts_tablet_manager.cc:1398] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.105288 27837 tablet_bootstrap.cc:492] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.105294 27838 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.105340 27837 ts_tablet_manager.cc:1367] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:07.105348 27838 ts_tablet_manager.cc:1367] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.105463 27837 raft_consensus.cc:348] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.105468 27838 raft_consensus.cc:348] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.105515 27837 raft_consensus.cc:374] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.105520 27838 raft_consensus.cc:374] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.105535 27837 raft_consensus.cc:729] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.105537 27838 raft_consensus.cc:729] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.105572 27838 consensus_queue.cc:260] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.105568 27837 consensus_queue.cc:260] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.105644 27838 ts_tablet_manager.cc:1398] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.105648 27837 ts_tablet_manager.cc:1398] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.105648 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.105691 27838 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.105706 27837 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.106084 27838 tablet_bootstrap.cc:654] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.106108 27837 tablet_bootstrap.cc:654] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.106160 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
W20240503 02:09:07.106236 27123 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 49a31c77c64c4261b5b02dd0acb9d9b0 consensus state has no leader
I20240503 02:09:07.106490 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.106597 27845 ts_tablet_manager.cc:1888] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.106634 27845 log.cc:1196] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a3bf2544755e4642803923e3f713f1e0
I20240503 02:09:07.106657 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.106773 27845 ts_tablet_manager.cc:1909] T a3bf2544755e4642803923e3f713f1e0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.106817 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.106894 27837 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.106930 27837 ts_tablet_manager.cc:1367] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.106987 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a3bf2544755e4642803923e3f713f1e0 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.107054 27837 raft_consensus.cc:348] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.107113 27837 raft_consensus.cc:374] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.107138 27837 raft_consensus.cc:729] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.107180 27837 consensus_queue.cc:260] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.107221 27839 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.107259 27837 ts_tablet_manager.cc:1398] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.107302 27837 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.107554 27839 tablet_bootstrap.cc:654] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.107625 27837 tablet_bootstrap.cc:654] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.107731 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
W20240503 02:09:07.107807 27123 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ee7d11b2bcc843aba268fe7007fba27d consensus state has no leader
I20240503 02:09:07.107990 27838 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.108036 27838 ts_tablet_manager.cc:1367] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.108117 27839 tablet_bootstrap.cc:492] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.108160 27839 ts_tablet_manager.cc:1367] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.108155 27838 raft_consensus.cc:348] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.108212 27838 raft_consensus.cc:374] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.108240 27838 raft_consensus.cc:729] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.108286 27838 consensus_queue.cc:260] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.108335 27839 raft_consensus.cc:348] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.108371 27838 ts_tablet_manager.cc:1398] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.108390 27839 raft_consensus.cc:374] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.108408 27839 raft_consensus.cc:729] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.108440 27839 consensus_queue.cc:260] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.108440 27838 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.108505 27839 ts_tablet_manager.cc:1398] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:07.108566 27839 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.108791 27838 tablet_bootstrap.cc:654] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.108811 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.108873 27839 tablet_bootstrap.cc:654] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.109133 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.109341 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]). Expected version 1 got 0
I20240503 02:09:07.109484 27837 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.109534 27837 ts_tablet_manager.cc:1367] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.109658 27837 raft_consensus.cc:348] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.109719 27837 raft_consensus.cc:374] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.109738 27837 raft_consensus.cc:729] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.109779 27837 consensus_queue.cc:260] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.109866 27837 ts_tablet_manager.cc:1398] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.109907 27839 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.109931 27837 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.109954 27839 ts_tablet_manager.cc:1367] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.109993 27838 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.110037 27838 ts_tablet_manager.cc:1367] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.110074 27839 raft_consensus.cc:348] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.110142 27839 raft_consensus.cc:374] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.110160 27839 raft_consensus.cc:729] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.110148 27838 raft_consensus.cc:348] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.110203 27838 raft_consensus.cc:374] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.110201 27839 consensus_queue.cc:260] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.110229 27838 raft_consensus.cc:729] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.110253 27837 tablet_bootstrap.cc:654] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.110268 27839 ts_tablet_manager.cc:1398] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.110276 27838 consensus_queue.cc:260] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.110327 27839 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.110352 27838 ts_tablet_manager.cc:1398] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.110639 27839 tablet_bootstrap.cc:654] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.110911 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.111037 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
W20240503 02:09:07.111083 27130 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 698f32409eba4b2f9c0b33ef4b76b494 consensus state has no leader
I20240503 02:09:07.111471 27130 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.111711 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.111794 27837 tablet_bootstrap.cc:492] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.111838 27837 ts_tablet_manager.cc:1367] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.111949 27837 raft_consensus.cc:348] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.111996 27837 raft_consensus.cc:374] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.112013 27837 raft_consensus.cc:729] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.112061 27837 consensus_queue.cc:260] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.112318 27837 ts_tablet_manager.cc:1398] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.112406 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.112740 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.112841 27839 tablet_bootstrap.cc:492] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.112881 27839 ts_tablet_manager.cc:1367] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.112998 27839 raft_consensus.cc:348] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.113039 27839 raft_consensus.cc:374] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.113055 27839 raft_consensus.cc:729] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.113093 27839 consensus_queue.cc:260] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.113181 27839 ts_tablet_manager.cc:1398] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.113502 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
I20240503 02:09:07.113903 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]). Expected version 1 got 0
W20240503 02:09:07.116389 27772 tablet.cc:2369] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240503 02:09:07.132722 27502 tablet.cc:2369] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 02:09:07.132977 27848 raft_consensus.cc:482] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.133040 27848 raft_consensus.cc:504] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.133293 27848 leader_election.cc:288] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.135692 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698f32409eba4b2f9c0b33ef4b76b494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.135795 27453 raft_consensus.cc:2454] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.135866 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698f32409eba4b2f9c0b33ef4b76b494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.135974 27723 raft_consensus.cc:2454] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.135984 27524 leader_election.cc:302] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.136080 27848 raft_consensus.cc:2781] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.136147 27848 raft_consensus.cc:482] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.136174 27848 raft_consensus.cc:3037] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.136837 27848 raft_consensus.cc:504] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.136933 27848 leader_election.cc:288] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.137064 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698f32409eba4b2f9c0b33ef4b76b494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.137109 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "698f32409eba4b2f9c0b33ef4b76b494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.137125 27723 raft_consensus.cc:3037] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.137152 27453 raft_consensus.cc:3037] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.137643 27453 raft_consensus.cc:2454] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.137706 27723 raft_consensus.cc:2454] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.137781 27524 leader_election.cc:302] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.137854 27848 raft_consensus.cc:2781] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.137954 27848 raft_consensus.cc:686] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.138010 27848 consensus_queue.cc:237] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.138623 27130 catalog_manager.cc:5484] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.138738 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.145743 27453 raft_consensus.cc:1261] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.145743 27723 raft_consensus.cc:1261] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.146005 27848 consensus_queue.cc:1035] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.146092 27884 consensus_queue.cc:1035] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.147171 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.147280 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.147889 27887 mvcc.cc:205] Tried to move back new op lower bound from 7023419994707345408 to 7023419994677817344. Current Snapshot: MvccSnapshot[applied={T|T < 7023419994707345408}]
I20240503 02:09:07.148000 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.147902 27892 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.148476 27891 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.148878 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.148979 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 1 got 0
I20240503 02:09:07.149065 27888 mvcc.cc:205] Tried to move back new op lower bound from 7023419994707345408 to 7023419994677817344. Current Snapshot: MvccSnapshot[applied={T|T < 7023419994707345408}]
I20240503 02:09:07.149309 27893 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.150550 27123 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_4f2a37398e784453af36b67aca62a69b [id=ffba5f39864e4bc29d61fd32c9f57d31] alter complete (version 1)
I20240503 02:09:07.180951 27853 raft_consensus.cc:482] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.181003 27853 raft_consensus.cc:504] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.181185 27853 leader_election.cc:288] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.183940 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd36ce35a6546f294c540534534253a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.184010 27453 raft_consensus.cc:2454] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.184064 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd36ce35a6546f294c540534534253a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.184140 27588 raft_consensus.cc:2454] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.184168 27659 leader_election.cc:302] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.184268 27853 raft_consensus.cc:2781] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.184314 27853 raft_consensus.cc:482] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.184340 27853 raft_consensus.cc:3037] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.184775 27853 raft_consensus.cc:504] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.184880 27853 leader_election.cc:288] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.185004 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd36ce35a6546f294c540534534253a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.185029 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdd36ce35a6546f294c540534534253a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.185065 27453 raft_consensus.cc:3037] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.185084 27588 raft_consensus.cc:3037] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.185498 27453 raft_consensus.cc:2454] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.185595 27588 raft_consensus.cc:2454] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.185652 27659 leader_election.cc:302] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.185725 27853 raft_consensus.cc:2781] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.185822 27853 raft_consensus.cc:686] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.185859 27853 consensus_queue.cc:237] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.186343 27130 catalog_manager.cc:5484] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.186426 27130 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2 [id=64630afa2665476592d50c89f1526567]). Expected version 1 got 0
I20240503 02:09:07.207473 27884 raft_consensus.cc:482] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.207553 27884 raft_consensus.cc:504] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.207648 27884 leader_election.cc:288] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.207813 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432c86ce274e474bb0366ef62dc7012b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.207829 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432c86ce274e474bb0366ef62dc7012b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.207875 27723 raft_consensus.cc:2454] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.207890 27453 raft_consensus.cc:2454] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.208016 27523 leader_election.cc:302] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.208115 27884 raft_consensus.cc:2781] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.208150 27884 raft_consensus.cc:482] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.208163 27884 raft_consensus.cc:3037] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.208638 27884 raft_consensus.cc:504] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.208739 27884 leader_election.cc:288] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.208875 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432c86ce274e474bb0366ef62dc7012b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.208899 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "432c86ce274e474bb0366ef62dc7012b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.208940 27453 raft_consensus.cc:3037] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.208947 27723 raft_consensus.cc:3037] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.209532 27723 raft_consensus.cc:2454] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.209532 27453 raft_consensus.cc:2454] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.209698 27523 leader_election.cc:302] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.209794 27884 raft_consensus.cc:2781] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.209836 27884 raft_consensus.cc:686] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.209892 27884 consensus_queue.cc:237] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.210427 27130 catalog_manager.cc:5484] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.210546 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.217355 27453 raft_consensus.cc:1261] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.217368 27723 raft_consensus.cc:1261] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.217542 27848 consensus_queue.cc:1035] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.217638 27884 consensus_queue.cc:1035] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.218612 27891 tablet.cc:1722] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.218665 27886 mvcc.cc:205] Tried to move back new op lower bound from 7023419995000807424 to 7023419994972016640. Current Snapshot: MvccSnapshot[applied={T|T < 7023419995000807424}]
I20240503 02:09:07.218724 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.218845 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.219102 27892 tablet.cc:1722] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.220682 27893 tablet.cc:1722] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.220790 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]). Expected version 1 got 0
I20240503 02:09:07.221055 27123 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_d79ff2033c254ab894e36b9a5d9652b4 [id=abfe5a29f62e4f2f884cf0e8ea66f56c] alter complete (version 1)
I20240503 02:09:07.235612 27130 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_1ed32182ffcb4fc79dea1f9de6085197"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.235637 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.235749 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.235749 27130 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ed32182ffcb4fc79dea1f9de6085197 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.235981 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_b5436c712b2d4efe8b257a09fdb6ef9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.236070 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5436c712b2d4efe8b257a09fdb6ef9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.236136 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_b23993840b064976bf9b999fbab05782"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.236133 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_854755e7631c444fbf6c264705849df7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.236264 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b23993840b064976bf9b999fbab05782 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.236320 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_854755e7631c444fbf6c264705849df7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.236934 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_ef7d98c2a6d34b4a9416df1897d18064"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.237025 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef7d98c2a6d34b4a9416df1897d18064 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.239084 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_f2ad1746d1bc42c3991e88f39e97dfae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.239053 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_bd953b4257d84ec38582c0ac7ce1df51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.239161 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2ad1746d1bc42c3991e88f39e97dfae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.239225 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd953b4257d84ec38582c0ac7ce1df51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.239277 27130 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_c78bc6eee8804364a5df93d733d4d5a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.239358 27130 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c78bc6eee8804364a5df93d733d4d5a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.239742 27123 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_d0da267ef6bb4ea49d811cf4fb9d89f2" } new_table_name: "default.Table_d313bd2d1c434bbc8d5d364828596066" modify_external_catalogs: true
I20240503 02:09:07.239782 27131 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_d79ff2033c254ab894e36b9a5d9652b4" } new_table_name: "default.Table_b9a02334455d47628918d830a41ba922" modify_external_catalogs: true
I20240503 02:09:07.240088 27129 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace" } new_table_name: "default.Table_cf88d339913e40869230650418817093" modify_external_catalogs: true
I20240503 02:09:07.240489 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4f2a37398e784453af36b67aca62a69b" } new_table_name: "default.Table_3e3b0d9c62d94690813b02f4646a0adf" modify_external_catalogs: true
I20240503 02:09:07.240648 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_1ed32182ffcb4fc79dea1f9de6085197" } new_table_name: "default.Table_dbc2c33b084947c7854b5676739b95df" modify_external_catalogs: true
I20240503 02:09:07.240792 27701 tablet_service.cc:1467] Processing CreateTablet for tablet faf4927166d74721a6d1890adadd3019 (DEFAULT_TABLE table=default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace [id=10fd8d0ff7104ffe99d73720ff9a5f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.240883 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf4927166d74721a6d1890adadd3019. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.241668 27129 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf4927166d74721a6d1890adadd3019 consensus state has no leader
I20240503 02:09:07.242005 27838 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.242368 27838 tablet_bootstrap.cc:654] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.241987 27125 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_cfdd82b518844e8eb778d0a3dad992bf" } modify_external_catalogs: true
I20240503 02:09:07.242445 27125 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_cfdd82b518844e8eb778d0a3dad992bf" } modify_external_catalogs: true
I20240503 02:09:07.242380 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_3f10232e10f94de69e51315eb0cd121b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.242717 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f10232e10f94de69e51315eb0cd121b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.242947 27838 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.242995 27838 ts_tablet_manager.cc:1367] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.243039 27568 tablet_service.cc:1467] Processing CreateTablet for tablet faf4927166d74721a6d1890adadd3019 (DEFAULT_TABLE table=default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace [id=10fd8d0ff7104ffe99d73720ff9a5f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.243132 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf4927166d74721a6d1890adadd3019. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.243124 27838 raft_consensus.cc:348] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.243166 27838 raft_consensus.cc:374] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.243178 27838 raft_consensus.cc:729] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.243211 27838 consensus_queue.cc:260] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.243284 27894 raft_consensus.cc:482] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.243324 27894 raft_consensus.cc:504] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.243641 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a31c77c64c4261b5b02dd0acb9d9b0" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.243289 27838 ts_tablet_manager.cc:1398] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.243705 27453 raft_consensus.cc:2454] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.243887 27659 leader_election.cc:302] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.243989 27130 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b23993840b064976bf9b999fbab05782" } modify_external_catalogs: true
I20240503 02:09:07.244041 27130 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b23993840b064976bf9b999fbab05782" } modify_external_catalogs: true
I20240503 02:09:07.244122 27894 leader_election.cc:288] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.244167 27894 raft_consensus.cc:2781] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.244187 27894 raft_consensus.cc:482] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.244203 27894 raft_consensus.cc:3037] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.244297 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a31c77c64c4261b5b02dd0acb9d9b0" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.244370 27588 raft_consensus.cc:2454] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.244429 27837 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.244735 27837 tablet_bootstrap.cc:654] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.244807 27894 raft_consensus.cc:504] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.245157 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a31c77c64c4261b5b02dd0acb9d9b0" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.245190 27894 leader_election.cc:288] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.245208 27453 raft_consensus.cc:3037] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.245707 27452 raft_consensus.cc:1261] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.245819 27453 raft_consensus.cc:2454] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.245968 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 957e479707bc4c25ae0284bd02031caa (DEFAULT_TABLE table=default.Table_b5436c712b2d4efe8b257a09fdb6ef9d [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.246054 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957e479707bc4c25ae0284bd02031caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.246181 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 957e479707bc4c25ae0284bd02031caa (DEFAULT_TABLE table=default.Table_b5436c712b2d4efe8b257a09fdb6ef9d [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.246234 27566 tablet_service.cc:1467] Processing CreateTablet for tablet c387e046acce47e6adbae30f9add354c (DEFAULT_TABLE table=default.Table_1ed32182ffcb4fc79dea1f9de6085197 [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.246271 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957e479707bc4c25ae0284bd02031caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.246320 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c387e046acce47e6adbae30f9add354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.246531 27564 tablet_service.cc:1467] Processing CreateTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 (DEFAULT_TABLE table=default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.246600 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d359e378054233aaf6ff6bccaa1a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.247197 27699 tablet_service.cc:1467] Processing CreateTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 (DEFAULT_TABLE table=default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.247270 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d359e378054233aaf6ff6bccaa1a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.247443 27837 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.247476 27837 ts_tablet_manager.cc:1367] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:07.247524 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a31c77c64c4261b5b02dd0acb9d9b0" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.247577 27588 raft_consensus.cc:3037] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.247612 27837 raft_consensus.cc:348] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.247663 27837 raft_consensus.cc:374] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.247680 27837 raft_consensus.cc:729] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.247718 27837 consensus_queue.cc:260] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.247797 27837 ts_tablet_manager.cc:1398] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.247856 27837 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.248122 27588 raft_consensus.cc:2454] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.248284 27837 tablet_bootstrap.cc:654] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.248443 27702 tablet_service.cc:1467] Processing CreateTablet for tablet c387e046acce47e6adbae30f9add354c (DEFAULT_TABLE table=default.Table_1ed32182ffcb4fc79dea1f9de6085197 [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.248525 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c387e046acce47e6adbae30f9add354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.248637 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 1903f7501d104c6789f55cced164bae2 (DEFAULT_TABLE table=default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.248721 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1903f7501d104c6789f55cced164bae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.249763 27587 raft_consensus.cc:1261] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.250324 27656 leader_election.cc:302] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
W20240503 02:09:07.250425 27122 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c387e046acce47e6adbae30f9add354c consensus state has no leader
I20240503 02:09:07.250572 27853 raft_consensus.cc:2781] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.250674 27853 raft_consensus.cc:686] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.250720 27853 consensus_queue.cc:237] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.250797 27838 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.250906 27853 consensus_queue.cc:1035] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.251029 27853 consensus_queue.cc:1035] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.251132 27838 tablet_bootstrap.cc:654] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.251462 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 1903f7501d104c6789f55cced164bae2 (DEFAULT_TABLE table=default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.251552 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1903f7501d104c6789f55cced164bae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.251904 27891 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20240503 02:09:07.252772 27637 tablet.cc:2369] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240503 02:09:07.253515 27429 tablet_service.cc:1467] Processing CreateTablet for tablet faf4927166d74721a6d1890adadd3019 (DEFAULT_TABLE table=default.Table_4fdc32ab2aea4e4b8a050c5cb9e09ace [id=10fd8d0ff7104ffe99d73720ff9a5f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.253525 27431 tablet_service.cc:1467] Processing CreateTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 (DEFAULT_TABLE table=default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.253624 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf4927166d74721a6d1890adadd3019. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.253741 27431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d359e378054233aaf6ff6bccaa1a56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.254652 27839 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.254671 27699 tablet_service.cc:1467] Processing CreateTablet for tablet f407188fe45843c49f00f9d3460756a3 (DEFAULT_TABLE table=default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.254679 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 1903f7501d104c6789f55cced164bae2 (DEFAULT_TABLE table=default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.254709 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 506cf6ad0d2645a491252b525117e9ad (DEFAULT_TABLE table=default.Table_bd953b4257d84ec38582c0ac7ce1df51 [id=737d279d213d4e598c098e4a32020f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.254747 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f407188fe45843c49f00f9d3460756a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.254757 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1903f7501d104c6789f55cced164bae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.254853 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 506cf6ad0d2645a491252b525117e9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.255120 27839 tablet_bootstrap.cc:654] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.255741 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 957e479707bc4c25ae0284bd02031caa (DEFAULT_TABLE table=default.Table_b5436c712b2d4efe8b257a09fdb6ef9d [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.255785 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.255813 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957e479707bc4c25ae0284bd02031caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.255859 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab67fe4e57d4d25830679b1848ca70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.256539 27433 tablet_service.cc:1467] Processing CreateTablet for tablet c387e046acce47e6adbae30f9add354c (DEFAULT_TABLE table=default.Table_1ed32182ffcb4fc79dea1f9de6085197 [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.256592 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c387e046acce47e6adbae30f9add354c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.257438 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.257645 27563 tablet_service.cc:1467] Processing CreateTablet for tablet f407188fe45843c49f00f9d3460756a3 (DEFAULT_TABLE table=default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.257735 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f407188fe45843c49f00f9d3460756a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.257797 27892 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.257903 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_43850e4338eb4b169fb5094bdf03eb07" } modify_external_catalogs: true
I20240503 02:09:07.257982 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_43850e4338eb4b169fb5094bdf03eb07" } modify_external_catalogs: true
I20240503 02:09:07.258066 27128 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_854755e7631c444fbf6c264705849df7" } modify_external_catalogs: true
I20240503 02:09:07.258108 27124 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b9a02334455d47628918d830a41ba922" } modify_external_catalogs: true
I20240503 02:09:07.258126 27128 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_854755e7631c444fbf6c264705849df7" } modify_external_catalogs: true
I20240503 02:09:07.258147 27124 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b9a02334455d47628918d830a41ba922" } modify_external_catalogs: true
I20240503 02:09:07.258204 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.258273 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8dd6a108b1430d80be6144bc6e7917. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.258558 27893 tablet.cc:1722] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.258891 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.258989 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab67fe4e57d4d25830679b1848ca70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.259331 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.259925 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 506cf6ad0d2645a491252b525117e9ad (DEFAULT_TABLE table=default.Table_bd953b4257d84ec38582c0ac7ce1df51 [id=737d279d213d4e598c098e4a32020f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.260025 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 506cf6ad0d2645a491252b525117e9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.260033 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 432c86ce274e474bb0366ef62dc7012b
I20240503 02:09:07.260107 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 9614b6572dfd443387ee3e4d676e8766 (DEFAULT_TABLE table=default.Table_3f10232e10f94de69e51315eb0cd121b [id=93d01889ff5b454e86796788b3862fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.260185 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9614b6572dfd443387ee3e4d676e8766. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.260035 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.261133 27837 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.261180 27837 ts_tablet_manager.cc:1367] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:07.261279 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 432c86ce274e474bb0366ef62dc7012b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.261413 27843 tablet_replica.cc:330] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.261477 27843 raft_consensus.cc:2229] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.261508 27843 pending_rounds.cc:62] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:07.261524 27843 pending_rounds.cc:68] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7023419995112140800 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "432c86ce274e474bb0366ef62dc7012b" schema { columns { id: 0 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_b9a02334455d47628918d830a41ba922" dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" new_extra_config { } }
I20240503 02:09:07.261616 27843 raft_consensus.cc:2258] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.261737 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.261744 27125 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet ee7d11b2bcc843aba268fe7007fba27d
I20240503 02:09:07.261296 27837 raft_consensus.cc:348] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.261932 27837 raft_consensus.cc:374] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.261961 27837 raft_consensus.cc:729] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.262007 27837 consensus_queue.cc:260] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.262102 27837 ts_tablet_manager.cc:1398] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.262140 27128 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 1903f7501d104c6789f55cced164bae2
I20240503 02:09:07.262529 27837 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:07.262580 27523 consensus_peers.cc:485] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Couldn't send request to peer c2be543b8aad4e80a58d6b4ab669faae. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:07.262802 27843 ts_tablet_manager.cc:1875] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.262923 27837 tablet_bootstrap.cc:654] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.263291 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet ee7d11b2bcc843aba268fe7007fba27d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.263375 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 1903f7501d104c6789f55cced164bae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.263425 27893 tablet.cc:1722] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.263947 27843 ts_tablet_manager.cc:1888] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.263988 27843 log.cc:1196] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/432c86ce274e474bb0366ef62dc7012b
I20240503 02:09:07.263686 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.264158 27843 ts_tablet_manager.cc:1909] T 432c86ce274e474bb0366ef62dc7012b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.264223 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8dd6a108b1430d80be6144bc6e7917. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.264693 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet e5d359e378054233aaf6ff6bccaa1a56
I20240503 02:09:07.265239 27838 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.265282 27838 ts_tablet_manager.cc:1367] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.000s
I20240503 02:09:07.265425 27838 raft_consensus.cc:348] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.265491 27838 raft_consensus.cc:374] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.265512 27838 raft_consensus.cc:729] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.265551 27838 consensus_queue.cc:260] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.265626 27838 ts_tablet_manager.cc:1398] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.265512 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.265684 27838 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.266064 27838 tablet_bootstrap.cc:654] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.266412 27839 tablet_bootstrap.cc:492] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.266456 27839 ts_tablet_manager.cc:1367] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240503 02:09:07.266515 27838 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.266546 27838 ts_tablet_manager.cc:1367] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.266577 27839 raft_consensus.cc:348] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.266631 27839 raft_consensus.cc:374] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.266647 27839 raft_consensus.cc:729] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.266662 27838 raft_consensus.cc:348] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.266683 27839 consensus_queue.cc:260] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.266705 27838 raft_consensus.cc:374] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.266722 27838 raft_consensus.cc:729] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.266749 27838 consensus_queue.cc:260] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.266759 27839 ts_tablet_manager.cc:1398] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.266772 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 1903f7501d104c6789f55cced164bae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.266829 27839 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.266839 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.266840 27838 ts_tablet_manager.cc:1398] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.266892 27838 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.266896 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet ee7d11b2bcc843aba268fe7007fba27d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.266938 27844 tablet_replica.cc:330] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.266999 27844 raft_consensus.cc:2229] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.267027 27844 raft_consensus.cc:2258] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.267169 27839 tablet_bootstrap.cc:654] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.267169 27838 tablet_bootstrap.cc:654] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.267205 27891 tablet.cc:1722] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.266939 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 432c86ce274e474bb0366ef62dc7012b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
W20240503 02:09:07.267374 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 1903f7501d104c6789f55cced164bae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1903f7501d104c6789f55cced164bae2 already in progress: creating tablet
W20240503 02:09:07.267437 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet e5d359e378054233aaf6ff6bccaa1a56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5d359e378054233aaf6ff6bccaa1a56 already in progress: creating tablet
I20240503 02:09:07.267472 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 9614b6572dfd443387ee3e4d676e8766 (DEFAULT_TABLE table=default.Table_3f10232e10f94de69e51315eb0cd121b [id=93d01889ff5b454e86796788b3862fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.267570 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9614b6572dfd443387ee3e4d676e8766. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.267613 27843 tablet_replica.cc:330] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.267658 27843 raft_consensus.cc:2229] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.267683 27843 raft_consensus.cc:2258] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.267700 27839 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.267733 27839 ts_tablet_manager.cc:1367] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.267863 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_b9a02334455d47628918d830a41ba922 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]) successfully deleted
I20240503 02:09:07.267865 27839 raft_consensus.cc:348] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.267920 27839 raft_consensus.cc:374] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.267938 27839 raft_consensus.cc:729] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.267941 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.267975 27839 consensus_queue.cc:260] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.268008 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.268050 27839 ts_tablet_manager.cc:1398] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.268095 27839 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.268400 27839 tablet_bootstrap.cc:654] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.268508 27843 ts_tablet_manager.cc:1875] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.268963 27431 tablet_service.cc:1467] Processing CreateTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.269052 27431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ab67fe4e57d4d25830679b1848ca70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.269177 27892 tablet.cc:1722] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.270040 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.270113 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f8dd6a108b1430d80be6144bc6e7917. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.270398 27892 tablet.cc:1717] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Skipping requested alter to schema version 2, tablet already version 2
I20240503 02:09:07.270959 27837 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.270998 27837 ts_tablet_manager.cc:1367] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:07.271018 27843 ts_tablet_manager.cc:1888] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.271016 27912 tablet.cc:1722] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.271061 27843 log.cc:1196] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ee7d11b2bcc843aba268fe7007fba27d
I20240503 02:09:07.271056 27432 tablet_service.cc:1467] Processing CreateTablet for tablet f407188fe45843c49f00f9d3460756a3 (DEFAULT_TABLE table=default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.271126 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f407188fe45843c49f00f9d3460756a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.271121 27837 raft_consensus.cc:348] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.271170 27837 raft_consensus.cc:374] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.271190 27837 raft_consensus.cc:729] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.271209 27843 ts_tablet_manager.cc:1909] T ee7d11b2bcc843aba268fe7007fba27d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.271226 27837 consensus_queue.cc:260] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.271306 27837 ts_tablet_manager.cc:1398] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.271351 27837 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.271507 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.271682 27837 tablet_bootstrap.cc:654] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.271845 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.271885 27130 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.272084 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 506cf6ad0d2645a491252b525117e9ad (DEFAULT_TABLE table=default.Table_bd953b4257d84ec38582c0ac7ce1df51 [id=737d279d213d4e598c098e4a32020f1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.272199 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 506cf6ad0d2645a491252b525117e9ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.272467 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 1903f7501d104c6789f55cced164bae2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1903f7501d104c6789f55cced164bae2 already in progress: creating tablet
I20240503 02:09:07.272558 27843 tablet_replica.cc:330] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.272603 27843 raft_consensus.cc:2229] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.272629 27843 raft_consensus.cc:2258] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.272734 27843 ts_tablet_manager.cc:1875] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.273129 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 9614b6572dfd443387ee3e4d676e8766 (DEFAULT_TABLE table=default.Table_3f10232e10f94de69e51315eb0cd121b [id=93d01889ff5b454e86796788b3862fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.273200 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9614b6572dfd443387ee3e4d676e8766. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.273715 27843 ts_tablet_manager.cc:1888] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.273756 27843 log.cc:1196] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e5d359e378054233aaf6ff6bccaa1a56
I20240503 02:09:07.273888 27843 ts_tablet_manager.cc:1909] T e5d359e378054233aaf6ff6bccaa1a56 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.274060 27917 tablet.cc:1722] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.274142 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet 432c86ce274e474bb0366ef62dc7012b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.274215 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.274266 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet ee7d11b2bcc843aba268fe7007fba27d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.274315 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 1903f7501d104c6789f55cced164bae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.274314 27843 tablet_replica.cc:330] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.274361 27838 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.274376 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.274374 27843 raft_consensus.cc:2229] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.274395 27838 ts_tablet_manager.cc:1367] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:07.274431 27843 pending_rounds.cc:62] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:07.274451 27843 pending_rounds.cc:68] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419995139158016 op_type: NO_OP noop_request { }
I20240503 02:09:07.274482 27843 raft_consensus.cc:2866] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.274523 27838 raft_consensus.cc:348] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.274581 27838 raft_consensus.cc:374] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.274598 27838 raft_consensus.cc:729] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.274619 27845 tablet_replica.cc:330] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.274621 27838 consensus_queue.cc:260] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.274665 27845 raft_consensus.cc:2229] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.274684 27838 ts_tablet_manager.cc:1398] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.274696 27845 raft_consensus.cc:2258] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.274740 27838 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.275040 27838 tablet_bootstrap.cc:654] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.275262 27130 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31] alter complete (version 2)
I20240503 02:09:07.275400 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e5d359e378054233aaf6ff6bccaa1a56 (table default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]) successfully deleted
I20240503 02:09:07.274515 27893 tablet.cc:1717] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Skipping requested alter to schema version 2, tablet already version 2
I20240503 02:09:07.275651 27844 ts_tablet_manager.cc:1875] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.276329 27838 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.276363 27838 ts_tablet_manager.cc:1367] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.276597 27838 raft_consensus.cc:348] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.276650 27838 raft_consensus.cc:374] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.276667 27838 raft_consensus.cc:729] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.276702 27838 consensus_queue.cc:260] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.276770 27838 ts_tablet_manager.cc:1398] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.276799 27844 ts_tablet_manager.cc:1888] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.276824 27838 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.276835 27844 log.cc:1196] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ee7d11b2bcc843aba268fe7007fba27d
I20240503 02:09:07.276991 27844 ts_tablet_manager.cc:1909] T ee7d11b2bcc843aba268fe7007fba27d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.277124 27838 tablet_bootstrap.cc:654] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.277383 27844 tablet_replica.cc:330] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.277447 27844 raft_consensus.cc:2229] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.277519 27844 raft_consensus.cc:2258] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.277644 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.277890 27838 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.277930 27838 ts_tablet_manager.cc:1367] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.278051 27838 raft_consensus.cc:348] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.278096 27838 raft_consensus.cc:374] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.278111 27838 raft_consensus.cc:729] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.278141 27838 consensus_queue.cc:260] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.278199 27838 ts_tablet_manager.cc:1398] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.278235 27838 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.278541 27838 tablet_bootstrap.cc:654] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.278815 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.278857 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.279453 27838 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.279505 27838 ts_tablet_manager.cc:1367] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.279846 27838 raft_consensus.cc:348] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.279903 27838 raft_consensus.cc:374] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.279922 27838 raft_consensus.cc:729] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.279946 27845 ts_tablet_manager.cc:1875] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.280083 27838 consensus_queue.cc:260] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.280553 27838 ts_tablet_manager.cc:1398] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.280856 27838 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.281126 27845 ts_tablet_manager.cc:1888] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.281167 27845 log.cc:1196] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/432c86ce274e474bb0366ef62dc7012b
I20240503 02:09:07.281191 27838 tablet_bootstrap.cc:654] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.281353 27845 ts_tablet_manager.cc:1909] T 432c86ce274e474bb0366ef62dc7012b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.281615 27128 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567] alter complete (version 2)
I20240503 02:09:07.281706 27845 tablet_replica.cc:330] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.281764 27845 raft_consensus.cc:2229] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.281790 27845 raft_consensus.cc:2258] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.281929 27845 ts_tablet_manager.cc:1875] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.282099 27844 ts_tablet_manager.cc:1875] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.282209 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_b9a02334455d47628918d830a41ba922 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]) successfully deleted
I20240503 02:09:07.283145 27844 ts_tablet_manager.cc:1888] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.283186 27844 log.cc:1196] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/432c86ce274e474bb0366ef62dc7012b
I20240503 02:09:07.283354 27844 ts_tablet_manager.cc:1909] T 432c86ce274e474bb0366ef62dc7012b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.283491 27845 ts_tablet_manager.cc:1888] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.283527 27845 log.cc:1196] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.283658 27845 ts_tablet_manager.cc:1909] T 49a31c77c64c4261b5b02dd0acb9d9b0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.274508 27843 raft_consensus.cc:2258] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.283874 27838 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.283928 27838 ts_tablet_manager.cc:1367] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.284121 27845 tablet_replica.cc:330] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.284109 27838 raft_consensus.cc:348] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.284185 27845 raft_consensus.cc:2229] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.284219 27845 raft_consensus.cc:2258] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.284242 27838 raft_consensus.cc:374] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.284340 27838 raft_consensus.cc:729] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.284386 27845 ts_tablet_manager.cc:1875] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.284458 27838 consensus_queue.cc:260] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.284503 27844 tablet_replica.cc:330] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.284519 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 432c86ce274e474bb0366ef62dc7012b (table default.Table_b9a02334455d47628918d830a41ba922 [id=abfe5a29f62e4f2f884cf0e8ea66f56c]) successfully deleted
I20240503 02:09:07.284543 27844 raft_consensus.cc:2229] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.284566 27844 raft_consensus.cc:2258] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.284725 27844 ts_tablet_manager.cc:1875] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.285251 27839 tablet_bootstrap.cc:492] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.285287 27839 ts_tablet_manager.cc:1367] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.000s
I20240503 02:09:07.285415 27839 raft_consensus.cc:348] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.285467 27839 raft_consensus.cc:374] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.285485 27839 raft_consensus.cc:729] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.285519 27839 consensus_queue.cc:260] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.285595 27839 ts_tablet_manager.cc:1398] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.285641 27839 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.285750 27844 ts_tablet_manager.cc:1888] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.285787 27844 log.cc:1196] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.285885 27845 ts_tablet_manager.cc:1888] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.285917 27844 ts_tablet_manager.cc:1909] T 49a31c77c64c4261b5b02dd0acb9d9b0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.285938 27845 log.cc:1196] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ee7d11b2bcc843aba268fe7007fba27d
I20240503 02:09:07.286099 27845 ts_tablet_manager.cc:1909] T ee7d11b2bcc843aba268fe7007fba27d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.286212 27838 ts_tablet_manager.cc:1398] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.285938 27839 tablet_bootstrap.cc:654] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.286451 27845 tablet_replica.cc:330] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.286497 27845 raft_consensus.cc:2229] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.286522 27845 raft_consensus.cc:2258] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.284729 27843 ts_tablet_manager.cc:1875] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.286671 27845 ts_tablet_manager.cc:1875] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.286682 27838 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.287102 27838 tablet_bootstrap.cc:654] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.287542 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.287663 27843 ts_tablet_manager.cc:1888] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.287703 27843 log.cc:1196] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.287863 27845 ts_tablet_manager.cc:1888] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.287884 27843 ts_tablet_manager.cc:1909] T 49a31c77c64c4261b5b02dd0acb9d9b0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.287909 27845 log.cc:1196] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1903f7501d104c6789f55cced164bae2
I20240503 02:09:07.288053 27845 ts_tablet_manager.cc:1909] T 1903f7501d104c6789f55cced164bae2 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.288386 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.289052 27837 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.289096 27837 ts_tablet_manager.cc:1367] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240503 02:09:07.289232 27838 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.289220 27837 raft_consensus.cc:348] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289273 27837 raft_consensus.cc:374] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.289276 27838 ts_tablet_manager.cc:1367] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.289290 27837 raft_consensus.cc:729] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.289348 27837 consensus_queue.cc:260] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289428 27837 ts_tablet_manager.cc:1398] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.289433 27839 tablet_bootstrap.cc:492] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.289469 27839 ts_tablet_manager.cc:1367] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.289484 27837 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.289487 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 49a31c77c64c4261b5b02dd0acb9d9b0 (table default.Table_43850e4338eb4b169fb5094bdf03eb07 [id=dd9422f4bcb74aaa8eafe96afede4d9f]) successfully deleted
I20240503 02:09:07.289530 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ee7d11b2bcc843aba268fe7007fba27d (table default.Table_cfdd82b518844e8eb778d0a3dad992bf [id=7d0f4b0dc4ae4f85b017fdd6cbbc41ca]) successfully deleted
I20240503 02:09:07.289556 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1903f7501d104c6789f55cced164bae2 (table default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]) successfully deleted
I20240503 02:09:07.289544 27838 raft_consensus.cc:348] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289597 27838 raft_consensus.cc:374] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 02:09:07.289606 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet e5d359e378054233aaf6ff6bccaa1a56 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5d359e378054233aaf6ff6bccaa1a56 already in progress: creating tablet
I20240503 02:09:07.289614 27838 raft_consensus.cc:729] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.289592 27839 raft_consensus.cc:348] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289638 27839 raft_consensus.cc:374] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.289656 27839 raft_consensus.cc:729] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.289642 27838 consensus_queue.cc:260] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289693 27839 consensus_queue.cc:260] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.289714 27838 ts_tablet_manager.cc:1398] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.289757 27838 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.289760 27839 ts_tablet_manager.cc:1398] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.289799 27837 tablet_bootstrap.cc:654] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.289810 27839 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.290061 27838 tablet_bootstrap.cc:654] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.290107 27839 tablet_bootstrap.cc:654] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.290840 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.290910 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.290997 27917 tablet.cc:1717] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Skipping requested alter to schema version 2, tablet already version 2
I20240503 02:09:07.291368 27837 tablet_bootstrap.cc:492] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.291401 27837 ts_tablet_manager.cc:1367] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.291535 27837 raft_consensus.cc:348] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.291591 27837 raft_consensus.cc:374] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.291610 27837 raft_consensus.cc:729] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
W20240503 02:09:07.291618 27117 catalog_manager.cc:4797] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 49a31c77c64c4261b5b02dd0acb9d9b0 because the tablet was not found. No further retry: Not found: Tablet not found: 49a31c77c64c4261b5b02dd0acb9d9b0
I20240503 02:09:07.291639 27837 consensus_queue.cc:260] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.291676 27844 tablet_replica.cc:330] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.291704 27837 ts_tablet_manager.cc:1398] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.291767 27837 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.292062 27837 tablet_bootstrap.cc:654] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.292162 27839 tablet_bootstrap.cc:492] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.292162 27130 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.292198 27839 ts_tablet_manager.cc:1367] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.292309 27839 raft_consensus.cc:348] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.292358 27839 raft_consensus.cc:374] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.292376 27839 raft_consensus.cc:729] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.292410 27839 consensus_queue.cc:260] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.292481 27839 ts_tablet_manager.cc:1398] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.292528 27839 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.292961 27839 tablet_bootstrap.cc:654] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.293045 27130 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f8dd6a108b1430d80be6144bc6e7917 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.293298 27128 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.291709 27844 raft_consensus.cc:2229] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.293532 27844 raft_consensus.cc:2258] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.293753 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 1903f7501d104c6789f55cced164bae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.293982 27837 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.294008 27843 tablet_replica.cc:330] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.294015 27837 ts_tablet_manager.cc:1367] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.294050 27843 raft_consensus.cc:2229] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.294077 27843 raft_consensus.cc:2258] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.294129 27837 raft_consensus.cc:348] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.294196 27837 raft_consensus.cc:374] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.294214 27837 raft_consensus.cc:729] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.294245 27837 consensus_queue.cc:260] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.294314 27837 ts_tablet_manager.cc:1398] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.293768 27838 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.294368 27837 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.294384 27838 ts_tablet_manager.cc:1367] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:07.294538 27838 raft_consensus.cc:348] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.294584 27838 raft_consensus.cc:374] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.294600 27838 raft_consensus.cc:729] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.294632 27838 consensus_queue.cc:260] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.294684 27837 tablet_bootstrap.cc:654] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.294701 27838 ts_tablet_manager.cc:1398] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.295120 27128 master_service.cc:946] ReplaceTablet: received request to replace tablet c387e046acce47e6adbae30f9add354c from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.295351 27839 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.295393 27839 ts_tablet_manager.cc:1367] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.295516 27839 raft_consensus.cc:348] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.295563 27839 raft_consensus.cc:374] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.295579 27839 raft_consensus.cc:729] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.295614 27839 consensus_queue.cc:260] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.295627 27125 master_service.cc:946] ReplaceTablet: received request to replace tablet 698f32409eba4b2f9c0b33ef4b76b494 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.295679 27839 ts_tablet_manager.cc:1398] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.295732 27839 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.296044 27839 tablet_bootstrap.cc:654] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.296175 27837 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.296208 27837 ts_tablet_manager.cc:1367] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.296304 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 1903f7501d104c6789f55cced164bae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.296319 27837 raft_consensus.cc:348] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.296370 27837 raft_consensus.cc:374] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.296386 27837 raft_consensus.cc:729] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.296417 27837 consensus_queue.cc:260] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.296435 27122 master_service.cc:946] ReplaceTablet: received request to replace tablet fdd36ce35a6546f294c540534534253a from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.296511 27837 ts_tablet_manager.cc:1398] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.296564 27837 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.296651 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 6f8dd6a108b1430d80be6144bc6e7917
I20240503 02:09:07.296722 27130 catalog_manager.cc:6554] ReplaceTablet: tablet 6f8dd6a108b1430d80be6144bc6e7917 deleted and replaced by tablet 8cbf03ee6db446169cdd24e91c11bdcd
I20240503 02:09:07.296926 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ab67fe4e57d4d25830679b1848ca70d from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.296990 27837 tablet_bootstrap.cc:654] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.297024 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cbf03ee6db446169cdd24e91c11bdcd) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.297199 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cbf03ee6db446169cdd24e91c11bdcd) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.297405 27848 raft_consensus.cc:482] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20240503 02:09:07.297451 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 6f8dd6a108b1430d80be6144bc6e7917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f8dd6a108b1430d80be6144bc6e7917 already in progress: creating tablet
I20240503 02:09:07.297446 27848 raft_consensus.cc:504] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.297554 27848 leader_election.cc:288] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.297720 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cbf03ee6db446169cdd24e91c11bdcd) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.297971 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957e479707bc4c25ae0284bd02031caa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.298032 27723 raft_consensus.cc:2454] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.298066 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957e479707bc4c25ae0284bd02031caa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.298115 27453 raft_consensus.cc:2454] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.298136 27523 leader_election.cc:302] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.298197 27848 raft_consensus.cc:2781] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.298224 27848 raft_consensus.cc:482] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.298243 27848 raft_consensus.cc:3037] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.298825 27848 raft_consensus.cc:504] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.298877 27843 ts_tablet_manager.cc:1875] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.298924 27848 leader_election.cc:288] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.299057 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957e479707bc4c25ae0284bd02031caa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.299104 27723 raft_consensus.cc:3037] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.299630 27723 raft_consensus.cc:2454] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.299757 27523 leader_election.cc:302] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.299916 27844 ts_tablet_manager.cc:1875] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.300091 27839 tablet_bootstrap.cc:492] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.300124 27839 ts_tablet_manager.cc:1367] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.300226 27128 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet c387e046acce47e6adbae30f9add354c
I20240503 02:09:07.300235 27839 raft_consensus.cc:348] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.300282 27839 raft_consensus.cc:374] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.300300 27839 raft_consensus.cc:729] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.300307 27128 catalog_manager.cc:6554] ReplaceTablet: tablet c387e046acce47e6adbae30f9add354c deleted and replaced by tablet af46440213c24b93b3c66fd2e5cd406f
I20240503 02:09:07.300338 27839 consensus_queue.cc:260] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.300407 27839 ts_tablet_manager.cc:1398] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.300459 27839 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.300669 27843 ts_tablet_manager.cc:1888] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.300709 27843 log.cc:1196] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1903f7501d104c6789f55cced164bae2
I20240503 02:09:07.300755 27839 tablet_bootstrap.cc:654] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.300846 27843 ts_tablet_manager.cc:1909] T 1903f7501d104c6789f55cced164bae2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.300928 27844 ts_tablet_manager.cc:1888] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.300967 27844 log.cc:1196] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e5d359e378054233aaf6ff6bccaa1a56
I20240503 02:09:07.301009 27843 tablet_replica.cc:330] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.301048 27843 raft_consensus.cc:2229] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.301071 27843 raft_consensus.cc:2258] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.301103 27844 ts_tablet_manager.cc:1909] T e5d359e378054233aaf6ff6bccaa1a56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.301191 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet c387e046acce47e6adbae30f9add354c with delete_type TABLET_DATA_DELETED (replaced by tablet af46440213c24b93b3c66fd2e5cd406f) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.301415 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957e479707bc4c25ae0284bd02031caa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.301483 27453 raft_consensus.cc:3037] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.301803 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1903f7501d104c6789f55cced164bae2 (table default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]) successfully deleted
I20240503 02:09:07.301878 27125 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 698f32409eba4b2f9c0b33ef4b76b494
I20240503 02:09:07.301934 27125 catalog_manager.cc:6554] ReplaceTablet: tablet 698f32409eba4b2f9c0b33ef4b76b494 deleted and replaced by tablet d212d041e0854d1b9cc872286ecbd6ea
I20240503 02:09:07.302040 27453 raft_consensus.cc:2454] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.302064 27844 tablet_replica.cc:330] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.302109 27844 raft_consensus.cc:2229] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.302102 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 with delete_type TABLET_DATA_DELETED (replaced by tablet d212d041e0854d1b9cc872286ecbd6ea) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.302131 27844 raft_consensus.cc:2258] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.302212 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet c387e046acce47e6adbae30f9add354c with delete_type TABLET_DATA_DELETED (replaced by tablet af46440213c24b93b3c66fd2e5cd406f) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.302240 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 with delete_type TABLET_DATA_DELETED (replaced by tablet d212d041e0854d1b9cc872286ecbd6ea) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.302274 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e5d359e378054233aaf6ff6bccaa1a56 (table default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]) successfully deleted
I20240503 02:09:07.302505 27839 tablet_bootstrap.cc:492] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.302542 27839 ts_tablet_manager.cc:1367] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.302657 27839 raft_consensus.cc:348] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.302700 27839 raft_consensus.cc:374] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.302716 27839 raft_consensus.cc:729] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.302747 27839 consensus_queue.cc:260] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.302835 27839 ts_tablet_manager.cc:1398] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.302832 27848 raft_consensus.cc:2781] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.302872 27848 raft_consensus.cc:686] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.302894 27839 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.302904 27848 consensus_queue.cc:237] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.302974 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet fdd36ce35a6546f294c540534534253a
I20240503 02:09:07.303035 27122 catalog_manager.cc:6554] ReplaceTablet: tablet fdd36ce35a6546f294c540534534253a deleted and replaced by tablet 4210a30264df45c89eec0a6bebe3b46c
I20240503 02:09:07.303222 27839 tablet_bootstrap.cc:654] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.303792 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.303875 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbf03ee6db446169cdd24e91c11bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.303905 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd36ce35a6546f294c540534534253a with delete_type TABLET_DATA_DELETED (replaced by tablet 4210a30264df45c89eec0a6bebe3b46c) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.304373 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet e5d359e378054233aaf6ff6bccaa1a56 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.304461 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet c387e046acce47e6adbae30f9add354c with delete_type TABLET_DATA_DELETED (replaced by tablet af46440213c24b93b3c66fd2e5cd406f) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.304517 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 698f32409eba4b2f9c0b33ef4b76b494 with delete_type TABLET_DATA_DELETED (replaced by tablet d212d041e0854d1b9cc872286ecbd6ea) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.304564 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd36ce35a6546f294c540534534253a with delete_type TABLET_DATA_DELETED (replaced by tablet 4210a30264df45c89eec0a6bebe3b46c) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.304615 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.304674 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbf03ee6db446169cdd24e91c11bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.304992 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd (DEFAULT_TABLE table=default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.305071 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbf03ee6db446169cdd24e91c11bdcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.305370 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 4210a30264df45c89eec0a6bebe3b46c (DEFAULT_TABLE table=default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.305370 27703 tablet_service.cc:1467] Processing CreateTablet for tablet af46440213c24b93b3c66fd2e5cd406f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.305437 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet c387e046acce47e6adbae30f9add354c with delete_type TABLET_DATA_DELETED (replaced by tablet af46440213c24b93b3c66fd2e5cd406f) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.305471 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af46440213c24b93b3c66fd2e5cd406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.305490 27700 tablet_service.cc:1467] Processing CreateTablet for tablet d212d041e0854d1b9cc872286ecbd6ea (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.305573 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d212d041e0854d1b9cc872286ecbd6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.306090 27838 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.306430 27838 tablet_bootstrap.cc:654] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.306442 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4210a30264df45c89eec0a6bebe3b46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.306480 27568 tablet_service.cc:1467] Processing CreateTablet for tablet af46440213c24b93b3c66fd2e5cd406f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.306530 27428 tablet_service.cc:1467] Processing CreateTablet for tablet af46440213c24b93b3c66fd2e5cd406f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.306565 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af46440213c24b93b3c66fd2e5cd406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.306610 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af46440213c24b93b3c66fd2e5cd406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.306823 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet fdd36ce35a6546f294c540534534253a with delete_type TABLET_DATA_DELETED (replaced by tablet 4210a30264df45c89eec0a6bebe3b46c) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.307169 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 4210a30264df45c89eec0a6bebe3b46c (DEFAULT_TABLE table=default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.307245 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4210a30264df45c89eec0a6bebe3b46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.306480 27564 tablet_service.cc:1467] Processing CreateTablet for tablet d212d041e0854d1b9cc872286ecbd6ea (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.307530 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d212d041e0854d1b9cc872286ecbd6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.307896 27424 tablet_service.cc:1467] Processing CreateTablet for tablet d212d041e0854d1b9cc872286ecbd6ea (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.307976 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d212d041e0854d1b9cc872286ecbd6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.308324 27839 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.308365 27839 ts_tablet_manager.cc:1367] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:07.308473 27839 raft_consensus.cc:348] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.308521 27839 raft_consensus.cc:374] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.308538 27839 raft_consensus.cc:729] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.308573 27839 consensus_queue.cc:260] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.308640 27839 ts_tablet_manager.cc:1398] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.308691 27839 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.309001 27839 tablet_bootstrap.cc:654] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.309103 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 4210a30264df45c89eec0a6bebe3b46c (DEFAULT_TABLE table=default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.309172 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4210a30264df45c89eec0a6bebe3b46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.309620 27125 catalog_manager.cc:5484] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.310276 27845 tablet_replica.cc:330] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.310313 27839 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.310323 27845 raft_consensus.cc:2229] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.310344 27839 ts_tablet_manager.cc:1367] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.310353 27845 raft_consensus.cc:2258] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.310458 27839 raft_consensus.cc:348] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.310510 27839 raft_consensus.cc:374] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.310528 27839 raft_consensus.cc:729] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.310590 27837 tablet_bootstrap.cc:492] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.310626 27837 ts_tablet_manager.cc:1367] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.000s
I20240503 02:09:07.310649 27839 consensus_queue.cc:260] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.310736 27837 raft_consensus.cc:348] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.310784 27837 raft_consensus.cc:374] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.310801 27837 raft_consensus.cc:729] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.310832 27837 consensus_queue.cc:260] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.310896 27837 ts_tablet_manager.cc:1398] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.310947 27837 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.311255 27837 tablet_bootstrap.cc:654] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.311378 27839 ts_tablet_manager.cc:1398] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.311983 27839 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.312042 27837 tablet_bootstrap.cc:492] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.312074 27837 ts_tablet_manager.cc:1367] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.312187 27837 raft_consensus.cc:348] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.312232 27837 raft_consensus.cc:374] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.312247 27837 raft_consensus.cc:729] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.312309 27839 tablet_bootstrap.cc:654] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.312297 27837 consensus_queue.cc:260] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.312685 27837 ts_tablet_manager.cc:1398] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.312935 27837 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.313359 27837 tablet_bootstrap.cc:654] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.313459 27838 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.313494 27838 ts_tablet_manager.cc:1367] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:07.313565 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 0ab67fe4e57d4d25830679b1848ca70d
I20240503 02:09:07.313606 27838 raft_consensus.cc:348] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.313642 27123 catalog_manager.cc:6554] ReplaceTablet: tablet 0ab67fe4e57d4d25830679b1848ca70d deleted and replaced by tablet 920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.313654 27838 raft_consensus.cc:374] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.313671 27838 raft_consensus.cc:729] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.313706 27838 consensus_queue.cc:260] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.313740 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d with delete_type TABLET_DATA_DELETED (replaced by tablet 920704b080df46dcb3fbb58646c34a1d) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.313777 27838 ts_tablet_manager.cc:1398] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.313838 27838 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.313895 27844 ts_tablet_manager.cc:1875] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.314239 27838 tablet_bootstrap.cc:654] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.315156 27844 ts_tablet_manager.cc:1888] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.315202 27844 log.cc:1196] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1903f7501d104c6789f55cced164bae2
I20240503 02:09:07.315339 27844 ts_tablet_manager.cc:1909] T 1903f7501d104c6789f55cced164bae2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.315402 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d with delete_type TABLET_DATA_DELETED (replaced by tablet 920704b080df46dcb3fbb58646c34a1d) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.315501 27844 tablet_replica.cc:330] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.315542 27844 raft_consensus.cc:2229] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.315555 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ab67fe4e57d4d25830679b1848ca70d with delete_type TABLET_DATA_DELETED (replaced by tablet 920704b080df46dcb3fbb58646c34a1d) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.315824 27838 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.315866 27838 ts_tablet_manager.cc:1367] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.315997 27838 raft_consensus.cc:348] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.316056 27838 raft_consensus.cc:374] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.316074 27838 raft_consensus.cc:729] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.316108 27838 consensus_queue.cc:260] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.316133 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1903f7501d104c6789f55cced164bae2 (table default.Table_854755e7631c444fbf6c264705849df7 [id=b5bdc9d4fb304ef0a5ca7b1953111f69]) successfully deleted
I20240503 02:09:07.316200 27838 ts_tablet_manager.cc:1398] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.316212 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f8dd6a108b1430d80be6144bc6e7917 with delete_type TABLET_DATA_DELETED (replaced by tablet 8cbf03ee6db446169cdd24e91c11bdcd) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.316253 27838 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.315567 27844 raft_consensus.cc:2258] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.316439 27837 tablet_bootstrap.cc:492] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.316465 27844 ts_tablet_manager.cc:1875] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.316478 27837 ts_tablet_manager.cc:1367] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.316573 27838 tablet_bootstrap.cc:654] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.316624 27837 raft_consensus.cc:348] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.316677 27837 raft_consensus.cc:374] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.316695 27837 raft_consensus.cc:729] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.316745 27837 consensus_queue.cc:260] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.317179 27837 ts_tablet_manager.cc:1398] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.317677 27837 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.317806 27844 ts_tablet_manager.cc:1888] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.317871 27844 log.cc:1196] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/6f8dd6a108b1430d80be6144bc6e7917
I20240503 02:09:07.318033 27844 ts_tablet_manager.cc:1909] T 6f8dd6a108b1430d80be6144bc6e7917 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.318166 27904 raft_consensus.cc:482] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.318215 27904 raft_consensus.cc:504] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.318300 27844 tablet_replica.cc:330] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.318336 27904 leader_election.cc:288] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.318346 27844 raft_consensus.cc:2229] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.318378 27844 raft_consensus.cc:2258] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.318637 27844 ts_tablet_manager.cc:1875] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.318743 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 920704b080df46dcb3fbb58646c34a1d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.318825 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920704b080df46dcb3fbb58646c34a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.318914 27839 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.318946 27839 ts_tablet_manager.cc:1367] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:07.319067 27839 raft_consensus.cc:348] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.319121 27839 raft_consensus.cc:374] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.319139 27839 raft_consensus.cc:729] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.319171 27839 consensus_queue.cc:260] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.319247 27839 ts_tablet_manager.cc:1398] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.319298 27839 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.319693 27844 ts_tablet_manager.cc:1888] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.319705 27839 tablet_bootstrap.cc:654] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.319741 27844 log.cc:1196] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c387e046acce47e6adbae30f9add354c
I20240503 02:09:07.319898 27844 ts_tablet_manager.cc:1909] T c387e046acce47e6adbae30f9add354c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.320051 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "506cf6ad0d2645a491252b525117e9ad" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.320111 27453 raft_consensus.cc:2454] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.320158 27844 tablet_replica.cc:330] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.320205 27844 raft_consensus.cc:2229] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.320257 27844 raft_consensus.cc:2258] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.320250 27659 leader_election.cc:302] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.320317 27904 raft_consensus.cc:2781] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.320340 27904 raft_consensus.cc:482] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.320355 27904 raft_consensus.cc:3037] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.320514 27844 ts_tablet_manager.cc:1875] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.320686 27838 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.320720 27838 ts_tablet_manager.cc:1367] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.320834 27838 raft_consensus.cc:348] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.320878 27838 raft_consensus.cc:374] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.320895 27838 raft_consensus.cc:729] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.320930 27838 consensus_queue.cc:260] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.320955 27904 raft_consensus.cc:504] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.321012 27838 ts_tablet_manager.cc:1398] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.321033 27904 leader_election.cc:288] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.321067 27838 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.321610 27844 ts_tablet_manager.cc:1888] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.321635 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "506cf6ad0d2645a491252b525117e9ad" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.321666 27844 log.cc:1196] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/698f32409eba4b2f9c0b33ef4b76b494
I20240503 02:09:07.321691 27453 raft_consensus.cc:3037] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.321774 27838 tablet_bootstrap.cc:654] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.321861 27844 ts_tablet_manager.cc:1909] T 698f32409eba4b2f9c0b33ef4b76b494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.322114 27844 tablet_replica.cc:330] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.322156 27844 raft_consensus.cc:2229] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.322190 27844 raft_consensus.cc:2258] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.322218 27453 raft_consensus.cc:2454] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.322237 27839 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.322265 27839 ts_tablet_manager.cc:1367] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.322293 27838 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.322332 27838 ts_tablet_manager.cc:1367] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.322381 27839 raft_consensus.cc:348] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.322408 27659 leader_election.cc:302] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.322433 27839 raft_consensus.cc:374] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.322450 27839 raft_consensus.cc:729] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.322470 27844 ts_tablet_manager.cc:1875] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.322482 27853 raft_consensus.cc:2781] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.322455 27838 raft_consensus.cc:348] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.322481 27839 consensus_queue.cc:260] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.322513 27853 raft_consensus.cc:686] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.322561 27853 consensus_queue.cc:237] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.322924 27839 ts_tablet_manager.cc:1398] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.323232 27839 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.323366 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 6f8dd6a108b1430d80be6144bc6e7917 (table default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:07.323436 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.323458 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "506cf6ad0d2645a491252b525117e9ad" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.323505 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.323518 27587 raft_consensus.cc:2454] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.323577 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 920704b080df46dcb3fbb58646c34a1d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.323659 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920704b080df46dcb3fbb58646c34a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.323951 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
W20240503 02:09:07.324029 27123 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet af46440213c24b93b3c66fd2e5cd406f consensus state has no leader
I20240503 02:09:07.323702 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 920704b080df46dcb3fbb58646c34a1d (DEFAULT_TABLE table=default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.324038 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.324116 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.324146 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920704b080df46dcb3fbb58646c34a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.324148 27125 catalog_manager.cc:5484] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.322513 27838 raft_consensus.cc:374] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.324296 27838 raft_consensus.cc:729] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.324331 27838 consensus_queue.cc:260] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.324409 27838 ts_tablet_manager.cc:1398] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.324542 27838 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.324657 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "506cf6ad0d2645a491252b525117e9ad" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.324715 27588 raft_consensus.cc:3037] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.325145 27838 tablet_bootstrap.cc:654] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.325338 27588 raft_consensus.cc:2454] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.325686 27844 ts_tablet_manager.cc:1888] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.325762 27844 log.cc:1196] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fdd36ce35a6546f294c540534534253a
I20240503 02:09:07.325820 27837 tablet_bootstrap.cc:654] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.326052 27844 ts_tablet_manager.cc:1909] T fdd36ce35a6546f294c540534534253a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.326206 27845 ts_tablet_manager.cc:1875] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.326335 27844 tablet_replica.cc:330] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.326371 27839 tablet_bootstrap.cc:654] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.326408 27844 raft_consensus.cc:2229] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.326440 27844 raft_consensus.cc:2258] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:07.326819 27125 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet d212d041e0854d1b9cc872286ecbd6ea consensus state has no leader
I20240503 02:09:07.327106 27844 ts_tablet_manager.cc:1875] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.327328 27845 ts_tablet_manager.cc:1888] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.327351 27838 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.327392 27838 ts_tablet_manager.cc:1367] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.327368 27839 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.327508 27839 ts_tablet_manager.cc:1367] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.327579 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.327558 27838 raft_consensus.cc:348] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.327615 27838 raft_consensus.cc:374] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.327637 27838 raft_consensus.cc:729] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.327625 27839 raft_consensus.cc:348] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.327675 27839 raft_consensus.cc:374] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.327693 27839 raft_consensus.cc:729] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.327687 27838 consensus_queue.cc:260] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.327739 27839 consensus_queue.cc:260] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.327755 27837 tablet_bootstrap.cc:492] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.327769 27838 ts_tablet_manager.cc:1398] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.327787 27837 ts_tablet_manager.cc:1367] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:07.327816 27839 ts_tablet_manager.cc:1398] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.327885 27839 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.328207 27839 tablet_bootstrap.cc:654] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.328159 27837 raft_consensus.cc:348] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.328338 27837 raft_consensus.cc:374] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.328356 27837 raft_consensus.cc:729] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.328357 27844 ts_tablet_manager.cc:1888] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.328426 27837 consensus_queue.cc:260] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.328450 27844 log.cc:1196] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0ab67fe4e57d4d25830679b1848ca70d
I20240503 02:09:07.328501 27837 ts_tablet_manager.cc:1398] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.328562 27837 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.328616 27844 ts_tablet_manager.cc:1909] T 0ab67fe4e57d4d25830679b1848ca70d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.328971 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
W20240503 02:09:07.329061 27125 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 4210a30264df45c89eec0a6bebe3b46c consensus state has no leader
I20240503 02:09:07.329216 27837 tablet_bootstrap.cc:654] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.329267 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0ab67fe4e57d4d25830679b1848ca70d (table default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.329460 27903 raft_consensus.cc:482] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.329519 27903 raft_consensus.cc:504] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.329636 27903 leader_election.cc:288] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.329679 27903 raft_consensus.cc:482] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.329710 27903 raft_consensus.cc:504] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.329775 27903 leader_election.cc:288] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.329775 27843 ts_tablet_manager.cc:1875] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.330119 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.330191 27452 raft_consensus.cc:2454] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.330372 27659 leader_election.cc:302] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.330461 27853 raft_consensus.cc:2781] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.330502 27853 raft_consensus.cc:482] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.330529 27853 raft_consensus.cc:3037] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.330875 27843 ts_tablet_manager.cc:1888] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.330930 27843 log.cc:1196] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/6f8dd6a108b1430d80be6144bc6e7917
I20240503 02:09:07.331080 27843 ts_tablet_manager.cc:1909] T 6f8dd6a108b1430d80be6144bc6e7917 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.331055 27853 raft_consensus.cc:504] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.331224 27853 leader_election.cc:288] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.331343 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f407188fe45843c49f00f9d3460756a3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.331441 27588 raft_consensus.cc:2454] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.331485 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.331386 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.331532 27452 raft_consensus.cc:3037] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.331624 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.331722 27843 tablet_replica.cc:330] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.331743 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f407188fe45843c49f00f9d3460756a3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.331773 27656 leader_election.cc:302] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.331794 27453 raft_consensus.cc:2454] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.331782 27843 raft_consensus.cc:2229] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.331853 27843 raft_consensus.cc:2258] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:07.331880 27656 leader_election.cc:341] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:07.331897 27904 raft_consensus.cc:2781] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.331923 27904 raft_consensus.cc:482] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.331938 27904 raft_consensus.cc:3037] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:07.331959 27656 leader_election.cc:341] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:07.332043 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 6f8dd6a108b1430d80be6144bc6e7917 (table default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:07.332047 27843 ts_tablet_manager.cc:1875] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.332173 27452 raft_consensus.cc:2454] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.332269 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.332466 27904 raft_consensus.cc:504] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.332520 27837 tablet_bootstrap.cc:492] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.332481 27659 leader_election.cc:302] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:07.332553 27837 ts_tablet_manager.cc:1367] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.332605 27904 leader_election.cc:288] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.332638 27904 raft_consensus.cc:482] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.332659 27904 raft_consensus.cc:504] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.332674 27837 raft_consensus.cc:348] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.332720 27837 raft_consensus.cc:374] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.332720 27904 leader_election.cc:288] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.332741 27837 raft_consensus.cc:729] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.332700 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f407188fe45843c49f00f9d3460756a3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.332748 27904 raft_consensus.cc:2781] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.332777 27904 raft_consensus.cc:686] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.332772 27452 raft_consensus.cc:3037] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.332794 27837 consensus_queue.cc:260] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.332805 27904 consensus_queue.cc:237] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.332864 27837 ts_tablet_manager.cc:1398] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.332917 27837 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.333165 27843 ts_tablet_manager.cc:1888] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.333213 27843 log.cc:1196] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c387e046acce47e6adbae30f9add354c
I20240503 02:09:07.333251 27837 tablet_bootstrap.cc:654] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.333251 27452 raft_consensus.cc:2454] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.333391 27843 ts_tablet_manager.cc:1909] T c387e046acce47e6adbae30f9add354c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.333499 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f407188fe45843c49f00f9d3460756a3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.333514 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf4927166d74721a6d1890adadd3019" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.333564 27586 raft_consensus.cc:3037] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.333595 27587 raft_consensus.cc:2454] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.333724 27656 leader_election.cc:302] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.333794 27853 raft_consensus.cc:2781] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.333834 27853 raft_consensus.cc:482] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.333863 27853 raft_consensus.cc:3037] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.334111 27837 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.334160 27837 ts_tablet_manager.cc:1367] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.334169 27586 raft_consensus.cc:2454] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.334313 27656 leader_election.cc:302] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.334331 27837 raft_consensus.cc:348] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.334385 27837 raft_consensus.cc:374] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.334359 27853 raft_consensus.cc:504] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.334391 27903 raft_consensus.cc:2781] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.334403 27837 raft_consensus.cc:729] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.334430 27903 raft_consensus.cc:686] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.334455 27853 leader_election.cc:288] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.334468 27903 consensus_queue.cc:237] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.334683 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf4927166d74721a6d1890adadd3019" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.334695 27837 consensus_queue.cc:260] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.334748 27452 raft_consensus.cc:2454] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.334794 27837 ts_tablet_manager.cc:1398] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.334841 27837 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.335289 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.335325 27843 tablet_replica.cc:330] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.335376 27843 raft_consensus.cc:2229] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.335357 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf4927166d74721a6d1890adadd3019" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.335410 27586 raft_consensus.cc:3037] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.335537 27837 tablet_bootstrap.cc:654] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.335588 27843 raft_consensus.cc:2258] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.335903 27586 raft_consensus.cc:2454] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.335919 27843 ts_tablet_manager.cc:1875] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.336048 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf4927166d74721a6d1890adadd3019" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.336113 27453 raft_consensus.cc:3037] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.336254 27845 log.cc:1196] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e5d359e378054233aaf6ff6bccaa1a56
I20240503 02:09:07.336409 27845 ts_tablet_manager.cc:1909] T e5d359e378054233aaf6ff6bccaa1a56 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.336594 27845 tablet_replica.cc:330] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.336715 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e5d359e378054233aaf6ff6bccaa1a56 (table default.Table_b23993840b064976bf9b999fbab05782 [id=6e93a8b667b14fca9709b46311731188]) successfully deleted
I20240503 02:09:07.336616 27453 raft_consensus.cc:2454] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.336833 27845 raft_consensus.cc:2229] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.336866 27845 raft_consensus.cc:2258] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.336871 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.336926 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.336944 27659 leader_election.cc:302] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.337006 27903 raft_consensus.cc:2781] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.337023 27845 ts_tablet_manager.cc:1875] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.337049 27903 raft_consensus.cc:686] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.337044 27843 ts_tablet_manager.cc:1888] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.337121 27843 log.cc:1196] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/698f32409eba4b2f9c0b33ef4b76b494
I20240503 02:09:07.337095 27903 consensus_queue.cc:237] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.337185 27125 catalog_manager.cc:5484] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.337275 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.337359 27843 ts_tablet_manager.cc:1909] T 698f32409eba4b2f9c0b33ef4b76b494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.337533 27843 tablet_replica.cc:330] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.337576 27843 raft_consensus.cc:2229] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.337639 27843 raft_consensus.cc:2258] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.337987 27843 ts_tablet_manager.cc:1875] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.338043 27845 ts_tablet_manager.cc:1888] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.338065 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.338083 27845 log.cc:1196] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c387e046acce47e6adbae30f9add354c
I20240503 02:09:07.338212 27845 ts_tablet_manager.cc:1909] T c387e046acce47e6adbae30f9add354c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.338817 27839 tablet_bootstrap.cc:492] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.338855 27839 ts_tablet_manager.cc:1367] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:07.338816 27125 catalog_manager.cc:5484] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.338908 27125 catalog_manager.cc:5484] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.338971 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.338963 27839 raft_consensus.cc:348] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.339021 27839 raft_consensus.cc:374] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.339038 27839 raft_consensus.cc:729] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.339026 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c387e046acce47e6adbae30f9add354c (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.339037 27843 ts_tablet_manager.cc:1888] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.339078 27843 log.cc:1196] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fdd36ce35a6546f294c540534534253a
I20240503 02:09:07.339075 27839 consensus_queue.cc:260] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.339104 27845 tablet_replica.cc:330] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.339140 27903 raft_consensus.cc:482] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.339150 27839 ts_tablet_manager.cc:1398] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.339152 27845 raft_consensus.cc:2229] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.339179 27903 raft_consensus.cc:504] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.339221 27839 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.339269 27903 leader_election.cc:288] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.339548 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.339610 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
W20240503 02:09:07.339815 27656 leader_election.cc:341] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:07.339219 27845 raft_consensus.cc:2258] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.339625 27839 tablet_bootstrap.cc:654] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.339771 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:07.340230 27659 leader_election.cc:341] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:07.340267 27659 leader_election.cc:302] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:07.340325 27903 raft_consensus.cc:2726] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:07.340365 27845 ts_tablet_manager.cc:1875] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.340606 27839 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.340646 27839 ts_tablet_manager.cc:1367] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.339272 27843 ts_tablet_manager.cc:1909] T fdd36ce35a6546f294c540534534253a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.340795 27839 raft_consensus.cc:348] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.340842 27839 raft_consensus.cc:374] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.340859 27839 raft_consensus.cc:729] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.340893 27839 consensus_queue.cc:260] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.340957 27843 tablet_replica.cc:330] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.340981 27839 ts_tablet_manager.cc:1398] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.340993 27843 raft_consensus.cc:2229] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.341017 27843 raft_consensus.cc:2258] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.341145 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.341437 27843 ts_tablet_manager.cc:1875] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.341673 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.342072 27837 tablet_bootstrap.cc:492] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.342121 27837 ts_tablet_manager.cc:1367] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:07.342227 27845 ts_tablet_manager.cc:1888] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.342249 27837 raft_consensus.cc:348] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.342283 27845 log.cc:1196] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/698f32409eba4b2f9c0b33ef4b76b494
I20240503 02:09:07.342295 27837 raft_consensus.cc:374] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.342320 27837 raft_consensus.cc:729] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.342358 27837 consensus_queue.cc:260] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.342430 27837 ts_tablet_manager.cc:1398] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.342501 27837 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.342471 27845 ts_tablet_manager.cc:1909] T 698f32409eba4b2f9c0b33ef4b76b494 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.342821 27837 tablet_bootstrap.cc:654] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.342979 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.343096 27843 ts_tablet_manager.cc:1888] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.343153 27845 tablet_replica.cc:330] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.343175 27843 log.cc:1196] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0ab67fe4e57d4d25830679b1848ca70d
I20240503 02:09:07.343168 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 698f32409eba4b2f9c0b33ef4b76b494 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.343194 27845 raft_consensus.cc:2229] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.343230 27845 raft_consensus.cc:2258] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.343248 27837 tablet_bootstrap.cc:492] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.343279 27837 ts_tablet_manager.cc:1367] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.343351 27843 ts_tablet_manager.cc:1909] T 0ab67fe4e57d4d25830679b1848ca70d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.343386 27837 raft_consensus.cc:348] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.343436 27837 raft_consensus.cc:374] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.343453 27837 raft_consensus.cc:729] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.343500 27837 consensus_queue.cc:260] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.343575 27837 ts_tablet_manager.cc:1398] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.343606 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0ab67fe4e57d4d25830679b1848ca70d (table default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.343868 27884 raft_consensus.cc:482] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.343938 27884 raft_consensus.cc:504] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.344055 27884 leader_election.cc:288] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.344197 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.344290 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af46440213c24b93b3c66fd2e5cd406f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.344426 27453 raft_consensus.cc:2379] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:07.343802 27845 ts_tablet_manager.cc:1875] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.344573 27524 leader_election.cc:302] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:07.344653 27884 raft_consensus.cc:3037] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.345185 27884 raft_consensus.cc:2726] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:07.345528 27453 raft_consensus.cc:1261] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.345566 27586 raft_consensus.cc:1261] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.345747 27903 consensus_queue.cc:1035] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.345836 27903 consensus_queue.cc:1035] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.346204 27845 ts_tablet_manager.cc:1888] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.346307 27845 log.cc:1196] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fdd36ce35a6546f294c540534534253a
I20240503 02:09:07.346627 27125 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.346905 27893 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.346937 27845 ts_tablet_manager.cc:1909] T fdd36ce35a6546f294c540534534253a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.347512 27845 tablet_replica.cc:330] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.347615 27845 raft_consensus.cc:2229] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.347664 27845 raft_consensus.cc:2258] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.347815 27845 ts_tablet_manager.cc:1875] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.347925 27917 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.348923 27912 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.349259 27845 ts_tablet_manager.cc:1888] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.349462 27845 log.cc:1196] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0ab67fe4e57d4d25830679b1848ca70d
I20240503 02:09:07.349478 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fdd36ce35a6546f294c540534534253a (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
W20240503 02:09:07.349551 27118 catalog_manager.cc:4797] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet c387e046acce47e6adbae30f9add354c because the tablet was not found. No further retry: Not found: Tablet not found: c387e046acce47e6adbae30f9add354c
I20240503 02:09:07.349678 27845 ts_tablet_manager.cc:1909] T 0ab67fe4e57d4d25830679b1848ca70d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.349691 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet faf4927166d74721a6d1890adadd3019 (table default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76]). Expected version 1 got 0
I20240503 02:09:07.350780 27125 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_cf88d339913e40869230650418817093 [id=10fd8d0ff7104ffe99d73720ff9a5f76] alter complete (version 1)
I20240503 02:09:07.351343 27845 tablet_replica.cc:330] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.351454 27845 raft_consensus.cc:2229] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.351449 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0ab67fe4e57d4d25830679b1848ca70d (table default.Table_f2ad1746d1bc42c3991e88f39e97dfae [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.351532 27845 raft_consensus.cc:2258] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.351673 27845 ts_tablet_manager.cc:1875] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.352609 27845 ts_tablet_manager.cc:1888] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.352656 27845 log.cc:1196] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/6f8dd6a108b1430d80be6144bc6e7917
I20240503 02:09:07.352790 27845 ts_tablet_manager.cc:1909] T 6f8dd6a108b1430d80be6144bc6e7917 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.353075 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 6f8dd6a108b1430d80be6144bc6e7917 (table default.Table_ef7d98c2a6d34b4a9416df1897d18064 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:07.364336 27853 raft_consensus.cc:482] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.364393 27853 raft_consensus.cc:504] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.364490 27853 leader_election.cc:288] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.364598 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbf03ee6db446169cdd24e91c11bdcd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.364666 27453 raft_consensus.cc:2454] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.364665 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbf03ee6db446169cdd24e91c11bdcd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.364737 27586 raft_consensus.cc:2454] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.364773 27659 leader_election.cc:302] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.364854 27853 raft_consensus.cc:2781] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.364897 27853 raft_consensus.cc:482] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.364929 27853 raft_consensus.cc:3037] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.365378 27853 raft_consensus.cc:504] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.365494 27853 leader_election.cc:288] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.365595 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbf03ee6db446169cdd24e91c11bdcd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.365633 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbf03ee6db446169cdd24e91c11bdcd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.365659 27453 raft_consensus.cc:3037] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.365676 27586 raft_consensus.cc:3037] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.366058 27586 raft_consensus.cc:2454] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.366102 27453 raft_consensus.cc:2454] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.366200 27656 leader_election.cc:302] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.366307 27853 raft_consensus.cc:2781] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.366371 27853 raft_consensus.cc:686] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.366408 27853 consensus_queue.cc:237] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.366916 27586 raft_consensus.cc:1261] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.366945 27453 raft_consensus.cc:1261] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.367050 27125 catalog_manager.cc:5484] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.367116 27853 consensus_queue.cc:1035] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.367420 27853 consensus_queue.cc:1035] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.368115 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.368477 27893 tablet.cc:1722] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.369069 27917 tablet.cc:1722] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.369421 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.369805 27912 tablet.cc:1722] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.370723 27125 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3] alter complete (version 1)
I20240503 02:09:07.424487 27853 raft_consensus.cc:482] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.424544 27853 raft_consensus.cc:504] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.424649 27853 leader_election.cc:288] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.424779 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d212d041e0854d1b9cc872286ecbd6ea" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.424784 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d212d041e0854d1b9cc872286ecbd6ea" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.424849 27586 raft_consensus.cc:2454] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.424849 27453 raft_consensus.cc:2454] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.424994 27656 leader_election.cc:302] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.425086 27853 raft_consensus.cc:2781] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.425129 27853 raft_consensus.cc:482] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.425148 27853 raft_consensus.cc:3037] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.425668 27853 raft_consensus.cc:504] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.425782 27853 leader_election.cc:288] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.425889 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d212d041e0854d1b9cc872286ecbd6ea" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.425908 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d212d041e0854d1b9cc872286ecbd6ea" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.425942 27453 raft_consensus.cc:3037] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.425959 27586 raft_consensus.cc:3037] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.426414 27453 raft_consensus.cc:2454] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.426419 27586 raft_consensus.cc:2454] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.426575 27659 leader_election.cc:302] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.426678 27853 raft_consensus.cc:2781] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.426746 27853 raft_consensus.cc:686] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.426790 27853 consensus_queue.cc:237] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.427248 27125 catalog_manager.cc:5484] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.427352 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.437309 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_60758211018b43539a9b3a39b728bd64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.437438 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60758211018b43539a9b3a39b728bd64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.437752 27453 raft_consensus.cc:1261] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.437798 27586 raft_consensus.cc:1261] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.437965 27894 consensus_queue.cc:1035] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.438056 27853 consensus_queue.cc:1035] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.438884 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_7a3d1471909649efb6bf2ddb710a07e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.438984 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.439014 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_27a8698301354d42818e5744644ede84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.439088 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27a8698301354d42818e5744644ede84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.439159 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
W20240503 02:09:07.438987 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a3d1471909649efb6bf2ddb710a07e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.439371 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.440006 27912 tablet.cc:1722] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.439992 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_307ac144366a49cea3bc45bd999c9ee2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.440050 27128 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
W20240503 02:09:07.440093 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_307ac144366a49cea3bc45bd999c9ee2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.440238 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_04df5089db594390a6d7b94423b5d246"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.440325 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04df5089db594390a6d7b94423b5d246 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.440636 27893 tablet.cc:1722] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.442405 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 (DEFAULT_TABLE table=default.Table_60758211018b43539a9b3a39b728bd64 [id=3fc74e77e2e34caa8d28d5428b5806ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.442500 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b7b16f82564b1ba88df6c72e41eb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.442482 27917 tablet.cc:1722] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.442636 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 (DEFAULT_TABLE table=default.Table_60758211018b43539a9b3a39b728bd64 [id=3fc74e77e2e34caa8d28d5428b5806ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.442637 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_b5b489bda1684ad9959bcccfeb3f9cd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.442718 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b7b16f82564b1ba88df6c72e41eb62. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.442719 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.442832 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_9017d82aa4984e29a21642db1dd0c595"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.442905 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9017d82aa4984e29a21642db1dd0c595 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.443001 27124 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_60758211018b43539a9b3a39b728bd64" } new_table_name: "default.Table_786b4055ce5140d08d17981aef2d77b2" modify_external_catalogs: true
I20240503 02:09:07.443306 27839 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.443418 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b5436c712b2d4efe8b257a09fdb6ef9d" } new_table_name: "default.Table_1810f6c6fb234272afd935488b0054cf" modify_external_catalogs: true
I20240503 02:09:07.443648 27839 tablet_bootstrap.cc:654] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.443821 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_bd953b4257d84ec38582c0ac7ce1df51" } new_table_name: "default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc" modify_external_catalogs: true
I20240503 02:09:07.443966 27837 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.444031 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 (DEFAULT_TABLE table=default.Table_60758211018b43539a9b3a39b728bd64 [id=3fc74e77e2e34caa8d28d5428b5806ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.444110 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97b7b16f82564b1ba88df6c72e41eb62. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.444283 27837 tablet_bootstrap.cc:654] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.445111 27838 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.445482 27838 tablet_bootstrap.cc:654] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.445624 27424 tablet_service.cc:1467] Processing CreateTablet for tablet fa37b879af0a4b26982515c82198d6c6 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.445724 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa37b879af0a4b26982515c82198d6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.445940 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_f74a594ab88f4c1899893062bdbf6bda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.446027 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f74a594ab88f4c1899893062bdbf6bda in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.446105 27700 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0c02292ac45838865c225d5c5dcde (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.446182 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0c02292ac45838865c225d5c5dcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.446175 27701 tablet_service.cc:1467] Processing CreateTablet for tablet fa37b879af0a4b26982515c82198d6c6 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.446211 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_f52e53ed821144adbb28ebf3404cd854"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.446280 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa37b879af0a4b26982515c82198d6c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.446283 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f52e53ed821144adbb28ebf3404cd854 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.446177 27703 tablet_service.cc:1467] Processing CreateTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 (DEFAULT_TABLE table=default.Table_04df5089db594390a6d7b94423b5d246 [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.446449 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9041d6c599a44b89f6b2f2cf55d7f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.446748 27428 tablet_service.cc:1467] Processing CreateTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 (DEFAULT_TABLE table=default.Table_04df5089db594390a6d7b94423b5d246 [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.446820 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9041d6c599a44b89f6b2f2cf55d7f70. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.447541 27124 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 97b7b16f82564b1ba88df6c72e41eb62 consensus state has no leader
I20240503 02:09:07.447831 27702 tablet_service.cc:1467] Processing CreateTablet for tablet fcb3a56cb32a47499d52173bb3981c59 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.447912 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb3a56cb32a47499d52173bb3981c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.448304 27839 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.448344 27839 ts_tablet_manager.cc:1367] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:07.448475 27839 raft_consensus.cc:348] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.448525 27839 raft_consensus.cc:374] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.448542 27839 raft_consensus.cc:729] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.448578 27839 consensus_queue.cc:260] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.448664 27839 ts_tablet_manager.cc:1398] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.448729 27839 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.449067 27839 tablet_bootstrap.cc:654] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.449303 27568 tablet_service.cc:1467] Processing CreateTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 (DEFAULT_TABLE table=default.Table_04df5089db594390a6d7b94423b5d246 [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.449527 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9041d6c599a44b89f6b2f2cf55d7f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.449313 27564 tablet_service.cc:1467] Processing CreateTablet for tablet fcb3a56cb32a47499d52173bb3981c59 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.449697 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb3a56cb32a47499d52173bb3981c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.449338 27566 tablet_service.cc:1467] Processing CreateTablet for tablet fa37b879af0a4b26982515c82198d6c6 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.449817 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa37b879af0a4b26982515c82198d6c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.450135 27429 tablet_service.cc:1467] Processing CreateTablet for tablet fcb3a56cb32a47499d52173bb3981c59 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.450212 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb3a56cb32a47499d52173bb3981c59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.449371 27563 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0c02292ac45838865c225d5c5dcde (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.450594 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0c02292ac45838865c225d5c5dcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.450887 27838 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.450923 27838 ts_tablet_manager.cc:1367] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:07.451047 27838 raft_consensus.cc:348] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.451097 27838 raft_consensus.cc:374] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.451115 27838 raft_consensus.cc:729] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.451162 27838 consensus_queue.cc:260] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.451220 27723 raft_consensus.cc:1261] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.451248 27838 ts_tablet_manager.cc:1398] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.451328 27838 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.451328 27839 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.451364 27839 ts_tablet_manager.cc:1367] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.451480 27839 raft_consensus.cc:348] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.451532 27839 raft_consensus.cc:374] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.451507 27884 consensus_queue.cc:1035] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.451550 27839 raft_consensus.cc:729] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.451584 27839 consensus_queue.cc:260] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.451642 27838 tablet_bootstrap.cc:654] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.451702 27839 ts_tablet_manager.cc:1398] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.451704 27453 raft_consensus.cc:1261] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.452086 27900 consensus_queue.cc:1035] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.452488 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_cf88d339913e40869230650418817093" } new_table_name: "default.Table_2691a07efb4c43e698e53947e6b90445" modify_external_catalogs: true
I20240503 02:09:07.452556 27838 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.452594 27838 ts_tablet_manager.cc:1367] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.451759 27839 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.452711 27838 raft_consensus.cc:348] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.452766 27838 raft_consensus.cc:374] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.452787 27838 raft_consensus.cc:729] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.452821 27838 consensus_queue.cc:260] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.452864 27917 tablet.cc:1722] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.452931 27838 ts_tablet_manager.cc:1398] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.452984 27838 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.452983 27839 tablet_bootstrap.cc:654] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.453027 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (DEFAULT_TABLE table=default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.453104 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.453397 27838 tablet_bootstrap.cc:654] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.453601 27128 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_04df5089db594390a6d7b94423b5d246" } new_table_name: "default.Table_fcd07966c2874d0bba75d6179733b56d" modify_external_catalogs: true
I20240503 02:09:07.453881 27703 tablet_service.cc:1467] Processing CreateTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab (DEFAULT_TABLE table=default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.453959 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2c180bd912148dfbb6cfc373d1a91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.454267 27838 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.454305 27838 ts_tablet_manager.cc:1367] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.454427 27838 raft_consensus.cc:348] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.454494 27838 raft_consensus.cc:374] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.454514 27838 raft_consensus.cc:729] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.454541 27838 consensus_queue.cc:260] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.454615 27838 ts_tablet_manager.cc:1398] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.454676 27838 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.454989 27838 tablet_bootstrap.cc:654] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.455096 27428 tablet_service.cc:1467] Processing CreateTablet for tablet d7b0c02292ac45838865c225d5c5dcde (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.455180 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b0c02292ac45838865c225d5c5dcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.455698 27424 tablet_service.cc:1467] Processing CreateTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab (DEFAULT_TABLE table=default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.455760 27839 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.455782 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2c180bd912148dfbb6cfc373d1a91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.455796 27839 ts_tablet_manager.cc:1367] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.455818 27701 tablet_service.cc:1467] Processing CreateTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee (DEFAULT_TABLE table=default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.455900 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5c2af6e3e448c797fd7cf7e41526ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.455912 27839 raft_consensus.cc:348] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.455960 27839 raft_consensus.cc:374] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.455976 27839 raft_consensus.cc:729] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.456012 27839 consensus_queue.cc:260] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.456089 27839 ts_tablet_manager.cc:1398] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.456135 27839 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.456259 27838 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.456293 27838 ts_tablet_manager.cc:1367] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.456400 27838 raft_consensus.cc:348] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.456449 27838 raft_consensus.cc:374] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.456465 27838 raft_consensus.cc:729] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.456456 27839 tablet_bootstrap.cc:654] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.456502 27838 consensus_queue.cc:260] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.456578 27838 ts_tablet_manager.cc:1398] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.456625 27838 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.456770 27893 tablet.cc:1722] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.456921 27838 tablet_bootstrap.cc:654] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.457053 27839 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.457087 27839 ts_tablet_manager.cc:1367] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.457199 27839 raft_consensus.cc:348] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.457248 27839 raft_consensus.cc:374] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.457264 27839 raft_consensus.cc:729] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.457291 27839 consensus_queue.cc:260] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.457407 27839 ts_tablet_manager.cc:1398] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.457406 27564 tablet_service.cc:1467] Processing CreateTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee (DEFAULT_TABLE table=default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.457456 27839 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.457468 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.457487 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5c2af6e3e448c797fd7cf7e41526ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.457767 27839 tablet_bootstrap.cc:654] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.457796 27838 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.457830 27838 ts_tablet_manager.cc:1367] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.457950 27838 raft_consensus.cc:348] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.458007 27838 raft_consensus.cc:374] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.458024 27838 raft_consensus.cc:729] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.458079 27838 consensus_queue.cc:260] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.458151 27838 ts_tablet_manager.cc:1398] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.458142 27566 tablet_service.cc:1467] Processing CreateTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 (DEFAULT_TABLE table=default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.458140 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.458200 27838 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.458216 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b60b4c2ba943f6b957d898f7ad6501. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:07.458282 27128 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f9041d6c599a44b89f6b2f2cf55d7f70 consensus state has no leader
I20240503 02:09:07.458451 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (DEFAULT_TABLE table=default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.458525 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.458563 27838 tablet_bootstrap.cc:654] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.458667 27128 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f9041d6c599a44b89f6b2f2cf55d7f70 (table default.Table_fcd07966c2874d0bba75d6179733b56d [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]). Expected version 1 got 0
I20240503 02:09:07.458704 27128 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]). Expected version 1 got 0
I20240503 02:09:07.458863 27428 tablet_service.cc:1467] Processing CreateTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee (DEFAULT_TABLE table=default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.458935 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed5c2af6e3e448c797fd7cf7e41526ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.459199 27586 raft_consensus.cc:1261] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.459333 27702 tablet_service.cc:1467] Processing CreateTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 (DEFAULT_TABLE table=default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.459415 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b60b4c2ba943f6b957d898f7ad6501. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.459491 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (DEFAULT_TABLE table=default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.459559 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.459574 27452 raft_consensus.cc:1261] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.460075 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.460122 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f9041d6c599a44b89f6b2f2cf55d7f70 (table default.Table_fcd07966c2874d0bba75d6179733b56d [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]). Expected version 1 got 0
I20240503 02:09:07.460301 27838 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.460301 27839 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.460336 27839 ts_tablet_manager.cc:1367] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.460336 27838 ts_tablet_manager.cc:1367] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.460458 27838 raft_consensus.cc:348] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.460458 27839 raft_consensus.cc:348] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.460505 27838 raft_consensus.cc:374] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.460513 27839 raft_consensus.cc:374] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.460523 27838 raft_consensus.cc:729] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.460532 27839 raft_consensus.cc:729] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.460558 27839 consensus_queue.cc:260] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.460563 27838 consensus_queue.cc:260] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.460629 27839 ts_tablet_manager.cc:1398] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.460629 27838 ts_tablet_manager.cc:1398] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.460605 27853 consensus_queue.cc:1035] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.460678 27838 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.460678 27839 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.460738 27848 consensus_queue.cc:1035] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.460848 27424 tablet_service.cc:1467] Processing CreateTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 (DEFAULT_TABLE table=default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.460924 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b60b4c2ba943f6b957d898f7ad6501. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.460999 27839 tablet_bootstrap.cc:654] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.461027 27838 tablet_bootstrap.cc:654] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.461892 27983 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.461926 27912 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.462098 27838 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.462132 27838 ts_tablet_manager.cc:1367] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.462247 27838 raft_consensus.cc:348] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.462280 27838 raft_consensus.cc:374] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.462291 27838 raft_consensus.cc:729] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.462319 27838 consensus_queue.cc:260] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.462386 27838 ts_tablet_manager.cc:1398] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.462445 27838 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.462677 27838 tablet_bootstrap.cc:654] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.463143 27838 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.463188 27838 ts_tablet_manager.cc:1367] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.463297 27838 raft_consensus.cc:348] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.463299 27125 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f74a594ab88f4c1899893062bdbf6bda" } modify_external_catalogs: true
I20240503 02:09:07.463349 27838 raft_consensus.cc:374] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.463356 27125 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f74a594ab88f4c1899893062bdbf6bda" } modify_external_catalogs: true
I20240503 02:09:07.463368 27838 raft_consensus.cc:729] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.463390 27838 consensus_queue.cc:260] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.463454 27838 ts_tablet_manager.cc:1398] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.463516 27838 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.463650 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_fcd07966c2874d0bba75d6179733b56d" } modify_external_catalogs: true
I20240503 02:09:07.463711 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_fcd07966c2874d0bba75d6179733b56d" } modify_external_catalogs: true
I20240503 02:09:07.463794 27127 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f52e53ed821144adbb28ebf3404cd854" } modify_external_catalogs: true
I20240503 02:09:07.463852 27127 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f52e53ed821144adbb28ebf3404cd854" } modify_external_catalogs: true
I20240503 02:09:07.463941 27838 tablet_bootstrap.cc:654] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.464190 27563 tablet_service.cc:1467] Processing CreateTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab (DEFAULT_TABLE table=default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.464273 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2c180bd912148dfbb6cfc373d1a91ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.464291 27124 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d] alter complete (version 1)
I20240503 02:09:07.464648 27125 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet a7b60b4c2ba943f6b957d898f7ad6501
I20240503 02:09:07.464709 27838 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.464762 27838 ts_tablet_manager.cc:1367] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.464901 27838 raft_consensus.cc:348] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.464953 27838 raft_consensus.cc:374] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.464974 27838 raft_consensus.cc:729] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.465003 27838 consensus_queue.cc:260] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.465085 27838 ts_tablet_manager.cc:1398] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.465191 27125 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c78bc6eee8804364a5df93d733d4d5a7" } modify_external_catalogs: true
I20240503 02:09:07.465245 27125 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c78bc6eee8804364a5df93d733d4d5a7" } modify_external_catalogs: true
I20240503 02:09:07.465297 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_86dfc92ff9444e1aab6ee1670c47639a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.465418 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86dfc92ff9444e1aab6ee1670c47639a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.465773 27124 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b5b489bda1684ad9959bcccfeb3f9cd5" } modify_external_catalogs: true
I20240503 02:09:07.465829 27124 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b5b489bda1684ad9959bcccfeb3f9cd5" } modify_external_catalogs: true
I20240503 02:09:07.465894 27837 tablet_bootstrap.cc:492] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.465934 27837 ts_tablet_manager.cc:1367] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.000s
I20240503 02:09:07.466064 27837 raft_consensus.cc:348] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.466114 27837 raft_consensus.cc:374] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.466130 27837 raft_consensus.cc:729] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.466162 27837 consensus_queue.cc:260] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.466233 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.466328 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.461943 27839 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.466436 27839 ts_tablet_manager.cc:1367] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:07.466466 27843 tablet_replica.cc:330] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.466545 27839 raft_consensus.cc:348] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.466593 27839 raft_consensus.cc:374] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.466610 27839 raft_consensus.cc:729] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.466512 27843 raft_consensus.cc:2229] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.466645 27839 consensus_queue.cc:260] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.466635 27843 raft_consensus.cc:2258] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.466733 27839 ts_tablet_manager.cc:1398] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.466796 27839 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.467002 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet ed5c2af6e3e448c797fd7cf7e41526ee
I20240503 02:09:07.467108 27839 tablet_bootstrap.cc:654] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.467394 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.467473 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]). Expected version 1 got 0
I20240503 02:09:07.467530 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
W20240503 02:09:07.467566 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet a7b60b4c2ba943f6b957d898f7ad6501 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7b60b4c2ba943f6b957d898f7ad6501 already in progress: creating tablet
I20240503 02:09:07.466257 27837 ts_tablet_manager.cc:1398] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.467674 27837 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.467803 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
W20240503 02:09:07.467993 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet ed5c2af6e3e448c797fd7cf7e41526ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed5c2af6e3e448c797fd7cf7e41526ee already in progress: creating tablet
I20240503 02:09:07.468019 27837 tablet_bootstrap.cc:654] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.468184 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.468400 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet f9041d6c599a44b89f6b2f2cf55d7f70
I20240503 02:09:07.468634 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.468636 27839 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.468722 27839 ts_tablet_manager.cc:1367] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.468863 27837 tablet_bootstrap.cc:492] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.468855 27839 raft_consensus.cc:348] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.468894 27837 ts_tablet_manager.cc:1367] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.468905 27839 raft_consensus.cc:374] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.468922 27839 raft_consensus.cc:729] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.468952 27839 consensus_queue.cc:260] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.468724 27845 tablet_replica.cc:330] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.469026 27839 ts_tablet_manager.cc:1398] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.468999 27837 raft_consensus.cc:348] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.469058 27837 raft_consensus.cc:374] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.469060 27845 raft_consensus.cc:2229] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.469077 27837 raft_consensus.cc:729] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.469092 27845 raft_consensus.cc:2258] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.469101 27837 consensus_queue.cc:260] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.469168 27837 ts_tablet_manager.cc:1398] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.469219 27837 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.469226 27912 tablet.cc:1722] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.469485 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.469561 27845 ts_tablet_manager.cc:1875] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.469851 27837 tablet_bootstrap.cc:654] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:07.469941 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet a7b60b4c2ba943f6b957d898f7ad6501 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7b60b4c2ba943f6b957d898f7ad6501 already in progress: creating tablet
I20240503 02:09:07.470011 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.470070 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.470677 27843 ts_tablet_manager.cc:1875] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.470744 27845 ts_tablet_manager.cc:1888] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.470795 27845 log.cc:1196] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f9041d6c599a44b89f6b2f2cf55d7f70
I20240503 02:09:07.470934 27845 ts_tablet_manager.cc:1909] T f9041d6c599a44b89f6b2f2cf55d7f70 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
W20240503 02:09:07.471418 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet f9041d6c599a44b89f6b2f2cf55d7f70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f9041d6c599a44b89f6b2f2cf55d7f70 already in progress: creating tablet
W20240503 02:09:07.471486 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet ed5c2af6e3e448c797fd7cf7e41526ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ed5c2af6e3e448c797fd7cf7e41526ee already in progress: creating tablet
I20240503 02:09:07.471557 27983 tablet.cc:1722] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.471757 27917 tablet.cc:1722] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.471797 27843 ts_tablet_manager.cc:1888] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.471839 27843 log.cc:1196] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a7b60b4c2ba943f6b957d898f7ad6501
I20240503 02:09:07.471973 27843 ts_tablet_manager.cc:1909] T a7b60b4c2ba943f6b957d898f7ad6501 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.472255 27891 tablet.cc:1722] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.472517 27852 raft_consensus.cc:482] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.472556 27852 raft_consensus.cc:504] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.472723 27837 tablet_bootstrap.cc:492] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.472769 27837 ts_tablet_manager.cc:1367] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.472782 27852 leader_election.cc:288] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.472882 27837 raft_consensus.cc:348] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.472930 27837 raft_consensus.cc:374] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.472949 27837 raft_consensus.cc:729] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.472983 27837 consensus_queue.cc:260] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.473053 27837 ts_tablet_manager.cc:1398] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.473095 27837 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.473624 27837 tablet_bootstrap.cc:654] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.474052 27839 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.474139 27837 tablet_bootstrap.cc:492] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.474176 27837 ts_tablet_manager.cc:1367] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.474292 27837 raft_consensus.cc:348] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.474337 27837 raft_consensus.cc:374] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.474364 27837 raft_consensus.cc:729] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.474400 27837 consensus_queue.cc:260] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.474481 27837 ts_tablet_manager.cc:1398] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.474534 27837 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.474531 27839 tablet_bootstrap.cc:654] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.474839 27837 tablet_bootstrap.cc:654] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.475322 27125 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet f407188fe45843c49f00f9d3460756a3
I20240503 02:09:07.475569 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet f407188fe45843c49f00f9d3460756a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.475616 27843 tablet_replica.cc:330] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.475630 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet f407188fe45843c49f00f9d3460756a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.475636 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet f407188fe45843c49f00f9d3460756a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.475661 27843 raft_consensus.cc:2229] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.475685 27843 raft_consensus.cc:2258] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.475765 27844 tablet_replica.cc:330] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.475811 27844 raft_consensus.cc:2229] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.475818 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a7b60b4c2ba943f6b957d898f7ad6501 (table default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]) successfully deleted
I20240503 02:09:07.475837 27844 raft_consensus.cc:2258] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.475876 27843 ts_tablet_manager.cc:1875] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.476187 27844 ts_tablet_manager.cc:1875] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.476910 27843 ts_tablet_manager.cc:1888] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.476954 27843 log.cc:1196] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ed5c2af6e3e448c797fd7cf7e41526ee
I20240503 02:09:07.477088 27843 ts_tablet_manager.cc:1909] T ed5c2af6e3e448c797fd7cf7e41526ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.477389 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa37b879af0a4b26982515c82198d6c6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.477452 27723 raft_consensus.cc:2454] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.477452 27122 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f] alter complete (version 1)
I20240503 02:09:07.477511 27843 tablet_replica.cc:330] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.477548 27843 raft_consensus.cc:2229] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.477581 27843 pending_rounds.cc:62] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:07.477596 27843 pending_rounds.cc:68] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419995482124288 op_type: NO_OP noop_request { }
I20240503 02:09:07.477624 27843 raft_consensus.cc:2866] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.477640 27843 raft_consensus.cc:2258] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.477663 27388 leader_election.cc:302] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.477696 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ed5c2af6e3e448c797fd7cf7e41526ee (table default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]) successfully deleted
I20240503 02:09:07.477814 27852 raft_consensus.cc:2781] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.477867 27843 ts_tablet_manager.cc:1875] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.477887 27852 raft_consensus.cc:482] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.477911 27852 raft_consensus.cc:3037] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.478370 27852 raft_consensus.cc:504] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.478472 27852 leader_election.cc:288] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.478623 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa37b879af0a4b26982515c82198d6c6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.478679 27723 raft_consensus.cc:3037] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.479204 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 38e7bfba1d2e4ad2a7fec409b5a06f98
I20240503 02:09:07.479211 27723 raft_consensus.cc:2454] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.479348 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 388e82752def4a31b3d8bec18a478273 (DEFAULT_TABLE table=default.Table_86dfc92ff9444e1aab6ee1670c47639a [id=38784dc3ff464c6a8152e37f4d330fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.479432 27904 raft_consensus.cc:482] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.479440 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.479471 27904 raft_consensus.cc:504] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.479511 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 388e82752def4a31b3d8bec18a478273 (DEFAULT_TABLE table=default.Table_86dfc92ff9444e1aab6ee1670c47639a [id=38784dc3ff464c6a8152e37f4d330fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.479535 27130 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_2691a07efb4c43e698e53947e6b90445 [id=10fd8d0ff7104ffe99d73720ff9a5f76] alter complete (version 2)
I20240503 02:09:07.479575 27904 leader_election.cc:288] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.479642 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.479643 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 388e82752def4a31b3d8bec18a478273 (DEFAULT_TABLE table=default.Table_86dfc92ff9444e1aab6ee1670c47639a [id=38784dc3ff464c6a8152e37f4d330fd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.479714 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e82752def4a31b3d8bec18a478273. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.479813 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.480091 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e82752def4a31b3d8bec18a478273. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.480840 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.480870 27838 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.480871 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9614b6572dfd443387ee3e4d676e8766" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.480933 27586 raft_consensus.cc:2454] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.481101 27388 leader_election.cc:302] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.481148 27838 tablet_bootstrap.cc:654] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.481186 27852 raft_consensus.cc:2781] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.481190 27656 leader_election.cc:302] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.481230 27852 raft_consensus.cc:686] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.481279 27904 raft_consensus.cc:2781] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.481304 27904 raft_consensus.cc:482] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.481276 27852 consensus_queue.cc:237] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.481338 27904 raft_consensus.cc:3037] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.481467 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]). Expected version 1 got 0
I20240503 02:09:07.481695 27843 ts_tablet_manager.cc:1888] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.481740 27843 log.cc:1196] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f407188fe45843c49f00f9d3460756a3
I20240503 02:09:07.481876 27904 raft_consensus.cc:504] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.481916 27843 ts_tablet_manager.cc:1909] T f407188fe45843c49f00f9d3460756a3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.481946 27904 leader_election.cc:288] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.482066 27843 tablet_replica.cc:330] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.482095 27843 raft_consensus.cc:2229] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.482115 27843 raft_consensus.cc:2258] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.482297 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f407188fe45843c49f00f9d3460756a3 (table default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]) successfully deleted
I20240503 02:09:07.482334 27843 ts_tablet_manager.cc:1875] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.482477 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9614b6572dfd443387ee3e4d676e8766" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.482533 27586 raft_consensus.cc:3037] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.482637 27122 catalog_manager.cc:5484] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.482743 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9614b6572dfd443387ee3e4d676e8766" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.482792 27452 raft_consensus.cc:3037] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.483096 27586 raft_consensus.cc:2454] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.483284 27452 raft_consensus.cc:2454] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.483341 27843 ts_tablet_manager.cc:1888] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.483382 27843 log.cc:1196] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/38e7bfba1d2e4ad2a7fec409b5a06f98
I20240503 02:09:07.483479 27838 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.483516 27838 ts_tablet_manager.cc:1367] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:07.483624 27837 tablet_bootstrap.cc:492] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.483651 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa37b879af0a4b26982515c82198d6c6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.483660 27838 raft_consensus.cc:348] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.483711 27587 raft_consensus.cc:3037] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.483707 27838 raft_consensus.cc:374] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.483657 27837 ts_tablet_manager.cc:1367] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:07.483742 27838 raft_consensus.cc:729] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.483772 27838 consensus_queue.cc:260] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.483846 27838 ts_tablet_manager.cc:1398] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.483865 27837 raft_consensus.cc:348] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.483924 27837 raft_consensus.cc:374] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.483942 27837 raft_consensus.cc:729] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.483983 27837 consensus_queue.cc:260] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.483518 27843 ts_tablet_manager.cc:1909] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.484057 27837 ts_tablet_manager.cc:1398] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.484115 27837 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.484297 27587 raft_consensus.cc:2454] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.484436 27837 tablet_bootstrap.cc:654] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.484548 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9614b6572dfd443387ee3e4d676e8766" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.484611 27453 raft_consensus.cc:2362] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:07.484737 27656 leader_election.cc:302] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.484818 27900 raft_consensus.cc:2781] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.484865 27900 raft_consensus.cc:686] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.483871 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa37b879af0a4b26982515c82198d6c6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.484910 27586 raft_consensus.cc:2362] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1. Re-sending same reply.
I20240503 02:09:07.484901 27900 consensus_queue.cc:237] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.485190 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (table default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]) successfully deleted
I20240503 02:09:07.485195 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.485208 27837 tablet_bootstrap.cc:492] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
W20240503 02:09:07.485257 27117 catalog_manager.cc:4797] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet a7b60b4c2ba943f6b957d898f7ad6501 because the tablet was not found. No further retry: Not found: Tablet not found: a7b60b4c2ba943f6b957d898f7ad6501
I20240503 02:09:07.485270 27837 ts_tablet_manager.cc:1367] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.485489 27837 raft_consensus.cc:348] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.485530 27839 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.485540 27837 raft_consensus.cc:374] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.485559 27837 raft_consensus.cc:729] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.485563 27839 ts_tablet_manager.cc:1367] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:07.485585 27837 consensus_queue.cc:260] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.485667 27837 ts_tablet_manager.cc:1398] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.485682 27839 raft_consensus.cc:348] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.485708 27837 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.485728 27839 raft_consensus.cc:374] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.485744 27839 raft_consensus.cc:729] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.485778 27839 consensus_queue.cc:260] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.485867 27839 ts_tablet_manager.cc:1398] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.485916 27839 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.485949 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.486037 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.486121 27837 tablet_bootstrap.cc:654] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.486147 27844 ts_tablet_manager.cc:1888] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.486187 27844 log.cc:1196] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f407188fe45843c49f00f9d3460756a3
I20240503 02:09:07.486207 27839 tablet_bootstrap.cc:654] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.486336 27844 ts_tablet_manager.cc:1909] T f407188fe45843c49f00f9d3460756a3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.486464 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e82752def4a31b3d8bec18a478273. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.486725 27123 catalog_manager.cc:5484] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.486933 27839 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.486968 27839 ts_tablet_manager.cc:1367] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.487084 27839 raft_consensus.cc:348] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.487133 27839 raft_consensus.cc:374] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.487151 27839 raft_consensus.cc:729] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.487193 27839 consensus_queue.cc:260] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.487286 27839 ts_tablet_manager.cc:1398] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.487326 27839 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:07.487365 27117 catalog_manager.cc:4797] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet ed5c2af6e3e448c797fd7cf7e41526ee because the tablet was not found. No further retry: Not found: Tablet not found: ed5c2af6e3e448c797fd7cf7e41526ee
I20240503 02:09:07.487520 27912 tablet.cc:1722] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.487649 27839 tablet_bootstrap.cc:654] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.487705 27845 tablet_replica.cc:330] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.487749 27845 raft_consensus.cc:2229] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.487773 27845 raft_consensus.cc:2258] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.487980 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f9041d6c599a44b89f6b2f2cf55d7f70 (table default.Table_fcd07966c2874d0bba75d6179733b56d [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]) successfully deleted
I20240503 02:09:07.487999 27844 tablet_replica.cc:330] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.488039 27844 raft_consensus.cc:2229] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.488062 27844 raft_consensus.cc:2258] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.488114 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f9041d6c599a44b89f6b2f2cf55d7f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.488196 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f407188fe45843c49f00f9d3460756a3 (table default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]) successfully deleted
W20240503 02:09:07.488236 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 already in progress: creating tablet
I20240503 02:09:07.488306 27844 ts_tablet_manager.cc:1875] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.488343 27843 tablet_replica.cc:330] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.488332 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet ed5c2af6e3e448c797fd7cf7e41526ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.488384 27843 raft_consensus.cc:2229] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.488408 27843 raft_consensus.cc:2258] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.488584 27843 ts_tablet_manager.cc:1875] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.489090 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.489342 27844 ts_tablet_manager.cc:1888] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.489384 27844 log.cc:1196] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f9041d6c599a44b89f6b2f2cf55d7f70
I20240503 02:09:07.489614 27844 ts_tablet_manager.cc:1909] T f9041d6c599a44b89f6b2f2cf55d7f70 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.489682 27843 ts_tablet_manager.cc:1888] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.489720 27843 log.cc:1196] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f9041d6c599a44b89f6b2f2cf55d7f70
I20240503 02:09:07.489846 27844 tablet_replica.cc:330] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.489884 27844 raft_consensus.cc:2229] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.489908 27844 raft_consensus.cc:2258] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.489903 27845 ts_tablet_manager.cc:1875] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.489920 27843 ts_tablet_manager.cc:1909] T f9041d6c599a44b89f6b2f2cf55d7f70 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.490024 27844 ts_tablet_manager.cc:1875] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.490943 27844 ts_tablet_manager.cc:1888] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.490967 27845 ts_tablet_manager.cc:1888] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.490986 27844 log.cc:1196] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ed5c2af6e3e448c797fd7cf7e41526ee
I20240503 02:09:07.491008 27845 log.cc:1196] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f407188fe45843c49f00f9d3460756a3
I20240503 02:09:07.491123 27844 ts_tablet_manager.cc:1909] T ed5c2af6e3e448c797fd7cf7e41526ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.491151 27845 ts_tablet_manager.cc:1909] T f407188fe45843c49f00f9d3460756a3 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.491331 27837 tablet_bootstrap.cc:492] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.491384 27837 ts_tablet_manager.cc:1367] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:07.491411 27845 tablet_replica.cc:330] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.491456 27845 raft_consensus.cc:2229] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.491482 27845 raft_consensus.cc:2258] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.491497 27837 raft_consensus.cc:348] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.491533 27839 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.491545 27837 raft_consensus.cc:374] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.491564 27837 raft_consensus.cc:729] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.491564 27839 ts_tablet_manager.cc:1367] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.491561 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f9041d6c599a44b89f6b2f2cf55d7f70 (table default.Table_fcd07966c2874d0bba75d6179733b56d [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]) successfully deleted
I20240503 02:09:07.491600 27837 consensus_queue.cc:260] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.491636 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f9041d6c599a44b89f6b2f2cf55d7f70 (table default.Table_fcd07966c2874d0bba75d6179733b56d [id=bc4b3a82376b4fc0a0bf5de8a8e9a8cb]) successfully deleted
I20240503 02:09:07.491679 27837 ts_tablet_manager.cc:1398] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.491664 27839 raft_consensus.cc:348] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.491716 27839 raft_consensus.cc:374] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.491725 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f407188fe45843c49f00f9d3460756a3 (table default.Table_c78bc6eee8804364a5df93d733d4d5a7 [id=1ae8dd2951414ec4818e9d008dfced3f]) successfully deleted
I20240503 02:09:07.491735 27839 raft_consensus.cc:729] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.491740 27837 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.491777 27839 consensus_queue.cc:260] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.491854 27839 ts_tablet_manager.cc:1398] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.492043 27837 tablet_bootstrap.cc:654] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.492048 27845 ts_tablet_manager.cc:1875] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.492528 27837 tablet_bootstrap.cc:492] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.492563 27837 ts_tablet_manager.cc:1367] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.492589 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ed5c2af6e3e448c797fd7cf7e41526ee (table default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]) successfully deleted
I20240503 02:09:07.492664 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b60b4c2ba943f6b957d898f7ad6501 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.492731 27844 tablet_replica.cc:330] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.492765 27844 raft_consensus.cc:2229] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.492787 27844 raft_consensus.cc:2258] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.492884 27844 ts_tablet_manager.cc:1875] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.493517 27845 ts_tablet_manager.cc:1888] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.493558 27845 log.cc:1196] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/38e7bfba1d2e4ad2a7fec409b5a06f98
I20240503 02:09:07.493652 27845 ts_tablet_manager.cc:1909] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.492681 27837 raft_consensus.cc:348] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.493814 27837 raft_consensus.cc:374] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.493839 27844 ts_tablet_manager.cc:1888] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.493844 27837 raft_consensus.cc:729] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.493923 27844 log.cc:1196] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a7b60b4c2ba943f6b957d898f7ad6501
I20240503 02:09:07.493945 27837 consensus_queue.cc:260] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.494052 27844 ts_tablet_manager.cc:1909] T a7b60b4c2ba943f6b957d898f7ad6501 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.494091 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (table default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]) successfully deleted
I20240503 02:09:07.494392 27837 ts_tablet_manager.cc:1398] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.494437 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a7b60b4c2ba943f6b957d898f7ad6501 (table default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]) successfully deleted
I20240503 02:09:07.494469 27837 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.494540 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.494601 27844 tablet_replica.cc:330] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.494632 27844 raft_consensus.cc:2229] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.494654 27844 raft_consensus.cc:2258] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.494836 27844 ts_tablet_manager.cc:1875] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.494912 27845 tablet_replica.cc:330] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.494964 27845 raft_consensus.cc:2229] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.494993 27845 raft_consensus.cc:2258] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.495014 27837 tablet_bootstrap.cc:654] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.495110 27845 ts_tablet_manager.cc:1875] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.495608 27837 tablet_bootstrap.cc:492] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.495656 27837 ts_tablet_manager.cc:1367] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.495798 27844 ts_tablet_manager.cc:1888] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.495780 27837 raft_consensus.cc:348] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.495851 27844 log.cc:1196] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/38e7bfba1d2e4ad2a7fec409b5a06f98
I20240503 02:09:07.495851 27837 raft_consensus.cc:374] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.495898 27837 raft_consensus.cc:729] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.495937 27837 consensus_queue.cc:260] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.496009 27844 ts_tablet_manager.cc:1909] T 38e7bfba1d2e4ad2a7fec409b5a06f98 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.496013 27837 ts_tablet_manager.cc:1398] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.496076 27837 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.496008 27845 ts_tablet_manager.cc:1888] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.496500 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 38e7bfba1d2e4ad2a7fec409b5a06f98 (table default.Table_b5b489bda1684ad9959bcccfeb3f9cd5 [id=6095c31401854a558035e35ba496c2d1]) successfully deleted
I20240503 02:09:07.496628 27884 raft_consensus.cc:482] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.496677 27884 raft_consensus.cc:504] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.496770 27884 leader_election.cc:288] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.496421 27845 log.cc:1196] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ed5c2af6e3e448c797fd7cf7e41526ee
I20240503 02:09:07.496866 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0c02292ac45838865c225d5c5dcde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.496883 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0c02292ac45838865c225d5c5dcde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.496923 27723 raft_consensus.cc:2454] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.496923 27453 raft_consensus.cc:2454] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.496990 27837 tablet_bootstrap.cc:654] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.497076 27845 ts_tablet_manager.cc:1909] T ed5c2af6e3e448c797fd7cf7e41526ee P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.497081 27523 leader_election.cc:302] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.497200 27884 raft_consensus.cc:2781] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.497233 27884 raft_consensus.cc:482] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.497257 27884 raft_consensus.cc:3037] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.497287 27845 tablet_replica.cc:330] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.497283 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ed5c2af6e3e448c797fd7cf7e41526ee (table default.Table_f52e53ed821144adbb28ebf3404cd854 [id=d0830618d73b4e0c949ccd863169ab8f]) successfully deleted
I20240503 02:09:07.497367 27845 raft_consensus.cc:2229] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.497404 27845 raft_consensus.cc:2258] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.497511 27845 ts_tablet_manager.cc:1875] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.497789 27884 raft_consensus.cc:504] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.497897 27884 leader_election.cc:288] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.498086 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0c02292ac45838865c225d5c5dcde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.498147 27723 raft_consensus.cc:3037] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.498353 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet d7b0c02292ac45838865c225d5c5dcde from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.498668 27723 raft_consensus.cc:2454] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.498736 27837 tablet_bootstrap.cc:492] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.498769 27837 ts_tablet_manager.cc:1367] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:07.498726 27845 ts_tablet_manager.cc:1888] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.498798 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b0c02292ac45838865c225d5c5dcde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.498931 27453 raft_consensus.cc:3037] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.498946 27837 raft_consensus.cc:348] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.499001 27837 raft_consensus.cc:374] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.499017 27837 raft_consensus.cc:729] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.499061 27523 leader_election.cc:302] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.499063 27837 consensus_queue.cc:260] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.499156 27884 raft_consensus.cc:2781] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.499213 27884 raft_consensus.cc:686] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.499270 27884 consensus_queue.cc:237] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.499214 27837 ts_tablet_manager.cc:1398] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.499500 27453 raft_consensus.cc:2454] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.498903 27845 log.cc:1196] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a7b60b4c2ba943f6b957d898f7ad6501
I20240503 02:09:07.499892 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet d7b0c02292ac45838865c225d5c5dcde
I20240503 02:09:07.499981 27123 catalog_manager.cc:6554] ReplaceTablet: tablet d7b0c02292ac45838865c225d5c5dcde deleted and replaced by tablet 9ff52b758952450dbc6eb3162457000f
I20240503 02:09:07.500135 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0c02292ac45838865c225d5c5dcde with delete_type TABLET_DATA_DELETED (replaced by tablet 9ff52b758952450dbc6eb3162457000f) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.500181 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0c02292ac45838865c225d5c5dcde with delete_type TABLET_DATA_DELETED (replaced by tablet 9ff52b758952450dbc6eb3162457000f) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.500236 27843 tablet_replica.cc:330] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.500275 27843 raft_consensus.cc:2229] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.500283 27845 ts_tablet_manager.cc:1909] T a7b60b4c2ba943f6b957d898f7ad6501 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.500300 27843 raft_consensus.cc:2258] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.500491 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0c02292ac45838865c225d5c5dcde with delete_type TABLET_DATA_DELETED (replaced by tablet 9ff52b758952450dbc6eb3162457000f) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.500553 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet d7b0c02292ac45838865c225d5c5dcde with delete_type TABLET_DATA_DELETED (replaced by tablet 9ff52b758952450dbc6eb3162457000f) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.500566 27844 tablet_replica.cc:330] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.500613 27844 raft_consensus.cc:2229] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.500648 27844 pending_rounds.cc:62] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:07.500664 27844 pending_rounds.cc:68] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419996157247488 op_type: NO_OP noop_request { }
I20240503 02:09:07.500710 27844 raft_consensus.cc:2866] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.500713 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a7b60b4c2ba943f6b957d898f7ad6501 (table default.Table_f74a594ab88f4c1899893062bdbf6bda [id=9e5ca83909934c5ab7b731ae3dbf27a1]) successfully deleted
I20240503 02:09:07.500730 27844 raft_consensus.cc:2258] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.500875 27845 tablet_replica.cc:330] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.501042 27843 ts_tablet_manager.cc:1875] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.501243 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff52b758952450dbc6eb3162457000f (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.501350 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff52b758952450dbc6eb3162457000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.501348 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff52b758952450dbc6eb3162457000f (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.501420 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff52b758952450dbc6eb3162457000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.501953 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff52b758952450dbc6eb3162457000f (DEFAULT_TABLE table=default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.502027 27845 raft_consensus.cc:2229] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.502056 27845 raft_consensus.cc:2258] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.502091 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff52b758952450dbc6eb3162457000f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.502194 27845 ts_tablet_manager.cc:1875] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.502537 27843 ts_tablet_manager.cc:1888] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.502578 27843 log.cc:1196] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d7b0c02292ac45838865c225d5c5dcde
I20240503 02:09:07.502718 27843 ts_tablet_manager.cc:1909] T d7b0c02292ac45838865c225d5c5dcde P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.503034 27839 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.503242 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet 957e479707bc4c25ae0284bd02031caa from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.503348 27845 ts_tablet_manager.cc:1888] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.503388 27845 log.cc:1196] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d7b0c02292ac45838865c225d5c5dcde
I20240503 02:09:07.503468 27839 tablet_bootstrap.cc:654] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.503590 27845 ts_tablet_manager.cc:1909] T d7b0c02292ac45838865c225d5c5dcde P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.503662 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d7b0c02292ac45838865c225d5c5dcde (table default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:07.503715 27838 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
W20240503 02:09:07.503844 28008 consensus_queue.cc:1159] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20240503 02:09:07.504009 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d7b0c02292ac45838865c225d5c5dcde (table default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:07.504050 27838 tablet_bootstrap.cc:654] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.504065 27844 ts_tablet_manager.cc:1875] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.504302 27839 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.504335 27839 ts_tablet_manager.cc:1367] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.504452 27839 raft_consensus.cc:348] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.504500 27839 raft_consensus.cc:374] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.504504 27838 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.504518 27839 raft_consensus.cc:729] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.504541 27838 ts_tablet_manager.cc:1367] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.504554 27839 consensus_queue.cc:260] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.504632 27839 ts_tablet_manager.cc:1398] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.504657 27838 raft_consensus.cc:348] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.504711 27838 raft_consensus.cc:374] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.504729 27838 raft_consensus.cc:729] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.504779 27838 consensus_queue.cc:260] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.504861 27838 ts_tablet_manager.cc:1398] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.505060 27844 ts_tablet_manager.cc:1888] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.505100 27844 log.cc:1196] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d7b0c02292ac45838865c225d5c5dcde
I20240503 02:09:07.505276 27844 ts_tablet_manager.cc:1909] T d7b0c02292ac45838865c225d5c5dcde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.505506 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 957e479707bc4c25ae0284bd02031caa
I20240503 02:09:07.505586 27123 catalog_manager.cc:6554] ReplaceTablet: tablet 957e479707bc4c25ae0284bd02031caa deleted and replaced by tablet f7430ba7aada43968a0aac0ac10c5f2d
I20240503 02:09:07.505718 27130 master_service.cc:946] ReplaceTablet: received request to replace tablet fa37b879af0a4b26982515c82198d6c6 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.506172 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 957e479707bc4c25ae0284bd02031caa with delete_type TABLET_DATA_DELETED (replaced by tablet f7430ba7aada43968a0aac0ac10c5f2d) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.506254 27843 tablet_replica.cc:330] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.506317 27843 raft_consensus.cc:2229] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.506322 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 957e479707bc4c25ae0284bd02031caa with delete_type TABLET_DATA_DELETED (replaced by tablet f7430ba7aada43968a0aac0ac10c5f2d) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.506348 27843 raft_consensus.cc:2258] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.506443 27845 tablet_replica.cc:330] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.506516 27845 raft_consensus.cc:2229] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.506561 27845 raft_consensus.cc:2258] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.506618 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d7b0c02292ac45838865c225d5c5dcde (table default.Table_7a3d1471909649efb6bf2ddb710a07e4 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:07.506639 27843 ts_tablet_manager.cc:1875] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.504065 27837 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:07.506685 27115 catalog_manager.cc:4797] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet d7b0c02292ac45838865c225d5c5dcde because the tablet was not found. No further retry: Not found: Tablet not found: d7b0c02292ac45838865c225d5c5dcde
I20240503 02:09:07.506673 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 957e479707bc4c25ae0284bd02031caa with delete_type TABLET_DATA_DELETED (replaced by tablet f7430ba7aada43968a0aac0ac10c5f2d) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.506736 27844 tablet_replica.cc:330] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.506772 27844 raft_consensus.cc:2229] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.506815 27844 raft_consensus.cc:2258] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.507004 27844 ts_tablet_manager.cc:1875] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.507155 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet fa37b879af0a4b26982515c82198d6c6
I20240503 02:09:07.507222 27130 catalog_manager.cc:6554] ReplaceTablet: tablet fa37b879af0a4b26982515c82198d6c6 deleted and replaced by tablet cbeba88b72524e9cb77720cb7ca13597
I20240503 02:09:07.507683 27843 ts_tablet_manager.cc:1888] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.507732 27843 log.cc:1196] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/957e479707bc4c25ae0284bd02031caa
I20240503 02:09:07.507777 27702 tablet_service.cc:1467] Processing CreateTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.507865 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7430ba7aada43968a0aac0ac10c5f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.507930 27843 ts_tablet_manager.cc:1909] T 957e479707bc4c25ae0284bd02031caa P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.508008 27844 ts_tablet_manager.cc:1888] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.508004 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet fa37b879af0a4b26982515c82198d6c6 with delete_type TABLET_DATA_DELETED (replaced by tablet cbeba88b72524e9cb77720cb7ca13597) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.508045 27844 log.cc:1196] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/957e479707bc4c25ae0284bd02031caa
I20240503 02:09:07.507139 27429 tablet_service.cc:1467] Processing CreateTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.508141 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7430ba7aada43968a0aac0ac10c5f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.508205 27844 ts_tablet_manager.cc:1909] T 957e479707bc4c25ae0284bd02031caa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.508322 27843 tablet_replica.cc:330] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.508360 27843 raft_consensus.cc:2229] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.508383 27843 raft_consensus.cc:2258] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.508535 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 957e479707bc4c25ae0284bd02031caa (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.508798 27843 ts_tablet_manager.cc:1875] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.508996 27838 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.507340 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet fa37b879af0a4b26982515c82198d6c6 with delete_type TABLET_DATA_DELETED (replaced by tablet cbeba88b72524e9cb77720cb7ca13597) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.509025 27839 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.509316 27838 tablet_bootstrap.cc:654] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.509385 27839 tablet_bootstrap.cc:654] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.509819 27838 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.509833 27843 ts_tablet_manager.cc:1888] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.509841 27839 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.509856 27838 ts_tablet_manager.cc:1367] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.509876 27843 log.cc:1196] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fa37b879af0a4b26982515c82198d6c6
I20240503 02:09:07.509884 27839 ts_tablet_manager.cc:1367] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.509985 27838 raft_consensus.cc:348] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.509994 27839 raft_consensus.cc:348] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.510037 27839 raft_consensus.cc:374] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.510051 27838 raft_consensus.cc:374] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.510054 27839 raft_consensus.cc:729] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.510069 27838 raft_consensus.cc:729] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.510089 27839 consensus_queue.cc:260] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.510102 27838 consensus_queue.cc:260] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.510164 27839 ts_tablet_manager.cc:1398] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.510169 27838 ts_tablet_manager.cc:1398] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.510403 27568 tablet_service.cc:1467] Processing CreateTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.510443 27566 tablet_service.cc:1467] Processing CreateTablet for tablet cbeba88b72524e9cb77720cb7ca13597 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.510490 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7430ba7aada43968a0aac0ac10c5f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.510494 27429 tablet_service.cc:1467] Processing CreateTablet for tablet cbeba88b72524e9cb77720cb7ca13597 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.510532 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeba88b72524e9cb77720cb7ca13597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.510561 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeba88b72524e9cb77720cb7ca13597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.510583 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 957e479707bc4c25ae0284bd02031caa (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.510012 27843 ts_tablet_manager.cc:1909] T fa37b879af0a4b26982515c82198d6c6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.511508 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet fa37b879af0a4b26982515c82198d6c6 with delete_type TABLET_DATA_DELETED (replaced by tablet cbeba88b72524e9cb77720cb7ca13597) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.511579 27130 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
W20240503 02:09:07.511648 27130 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f7430ba7aada43968a0aac0ac10c5f2d consensus state has no leader
I20240503 02:09:07.511758 27130 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.511968 27839 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.511999 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fa37b879af0a4b26982515c82198d6c6 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.512292 27839 tablet_bootstrap.cc:654] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.512337 27702 tablet_service.cc:1467] Processing CreateTablet for tablet cbeba88b72524e9cb77720cb7ca13597 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.512372 27130 master_service.cc:946] ReplaceTablet: received request to replace tablet d212d041e0854d1b9cc872286ecbd6ea from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.512418 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbeba88b72524e9cb77720cb7ca13597. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.512531 27844 tablet_replica.cc:330] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.512573 27844 raft_consensus.cc:2229] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.512598 27844 raft_consensus.cc:2258] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.512626 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.512759 27844 ts_tablet_manager.cc:1875] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.513034 27839 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.513070 27839 ts_tablet_manager.cc:1367] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.513162 27839 raft_consensus.cc:348] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.513206 27839 raft_consensus.cc:374] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.513226 27839 raft_consensus.cc:729] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.513262 27839 consensus_queue.cc:260] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.513401 27838 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.513458 27839 ts_tablet_manager.cc:1398] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.513722 27838 tablet_bootstrap.cc:654] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.513871 27844 ts_tablet_manager.cc:1888] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.513909 27844 log.cc:1196] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fa37b879af0a4b26982515c82198d6c6
I20240503 02:09:07.514040 27844 ts_tablet_manager.cc:1909] T fa37b879af0a4b26982515c82198d6c6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.514366 27837 tablet_bootstrap.cc:654] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.514436 27838 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.514472 27838 ts_tablet_manager.cc:1367] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.514582 27838 raft_consensus.cc:348] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.514627 27838 raft_consensus.cc:374] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.514645 27838 raft_consensus.cc:729] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.514684 27838 consensus_queue.cc:260] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.514757 27838 ts_tablet_manager.cc:1398] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.514803 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fa37b879af0a4b26982515c82198d6c6 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.515012 27837 tablet_bootstrap.cc:492] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.515058 27837 ts_tablet_manager.cc:1367] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.000s
I20240503 02:09:07.515182 27837 raft_consensus.cc:348] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.515255 27837 raft_consensus.cc:374] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.515291 27837 raft_consensus.cc:729] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.515339 27837 consensus_queue.cc:260] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.515434 27837 ts_tablet_manager.cc:1398] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.515558 27837 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.515818 27845 ts_tablet_manager.cc:1875] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.515888 27837 tablet_bootstrap.cc:654] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.516474 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet d212d041e0854d1b9cc872286ecbd6ea
I20240503 02:09:07.516551 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet fcb3a56cb32a47499d52173bb3981c59 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.516705 27852 raft_consensus.cc:482] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.516762 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet d212d041e0854d1b9cc872286ecbd6ea with delete_type TABLET_DATA_DELETED (replaced by tablet fe6fb5025f5847f8bb3617c22bdf8c66) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.516744 27852 raft_consensus.cc:504] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.516558 27130 catalog_manager.cc:6554] ReplaceTablet: tablet d212d041e0854d1b9cc872286ecbd6ea deleted and replaced by tablet fe6fb5025f5847f8bb3617c22bdf8c66
I20240503 02:09:07.516794 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet d212d041e0854d1b9cc872286ecbd6ea with delete_type TABLET_DATA_DELETED (replaced by tablet fe6fb5025f5847f8bb3617c22bdf8c66) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.516865 27852 leader_election.cc:288] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.516961 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb3a56cb32a47499d52173bb3981c59" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.517004 27586 raft_consensus.cc:2454] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.516919 27844 tablet_replica.cc:330] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.517033 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb3a56cb32a47499d52173bb3981c59" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.517069 27844 raft_consensus.cc:2229] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.517087 27723 raft_consensus.cc:2454] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.517132 27844 raft_consensus.cc:2258] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.517132 27386 leader_election.cc:302] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.517227 27852 raft_consensus.cc:2781] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.517264 27852 raft_consensus.cc:482] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.517283 27852 raft_consensus.cc:3037] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.517401 27844 ts_tablet_manager.cc:1875] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.517918 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet d212d041e0854d1b9cc872286ecbd6ea with delete_type TABLET_DATA_DELETED (replaced by tablet fe6fb5025f5847f8bb3617c22bdf8c66) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.517897 27852 raft_consensus.cc:504] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.518014 27852 leader_election.cc:288] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.518179 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb3a56cb32a47499d52173bb3981c59" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.518257 27723 raft_consensus.cc:3037] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.518582 27843 tablet_replica.cc:330] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.518590 27429 tablet_service.cc:1467] Processing CreateTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.518635 27843 raft_consensus.cc:2229] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.518671 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe6fb5025f5847f8bb3617c22bdf8c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.518685 27843 raft_consensus.cc:2258] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.518803 27723 raft_consensus.cc:2454] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.518985 27388 leader_election.cc:302] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.519121 27852 raft_consensus.cc:2781] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.519121 27843 ts_tablet_manager.cc:1875] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.519160 27852 raft_consensus.cc:686] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.519207 27852 consensus_queue.cc:237] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.519573 27839 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.519984 27839 tablet_bootstrap.cc:654] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.520207 27702 tablet_service.cc:1467] Processing CreateTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.520287 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe6fb5025f5847f8bb3617c22bdf8c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.520442 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet fcb3a56cb32a47499d52173bb3981c59
I20240503 02:09:07.520529 27123 catalog_manager.cc:6554] ReplaceTablet: tablet fcb3a56cb32a47499d52173bb3981c59 deleted and replaced by tablet 2ca063a19a044e6c8466598f44316ff5
I20240503 02:09:07.520608 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb3a56cb32a47499d52173bb3981c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ca063a19a044e6c8466598f44316ff5) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.520748 27843 ts_tablet_manager.cc:1888] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.520797 27843 log.cc:1196] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d212d041e0854d1b9cc872286ecbd6ea
I20240503 02:09:07.520970 27843 ts_tablet_manager.cc:1909] T d212d041e0854d1b9cc872286ecbd6ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.521061 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb3a56cb32a47499d52173bb3981c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ca063a19a044e6c8466598f44316ff5) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.521147 27843 tablet_replica.cc:330] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.521188 27843 raft_consensus.cc:2229] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.521211 27843 raft_consensus.cc:2258] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.521400 27838 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.521417 27566 tablet_service.cc:1467] Processing CreateTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (DEFAULT_TABLE table=default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.521494 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.521521 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb3a56cb32a47499d52173bb3981c59" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.521562 27586 raft_consensus.cc:3037] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.521703 27838 tablet_bootstrap.cc:654] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.521944 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 2ca063a19a044e6c8466598f44316ff5 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.521986 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 2ca063a19a044e6c8466598f44316ff5 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.522063 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ca063a19a044e6c8466598f44316ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.522094 27586 raft_consensus.cc:2454] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.522167 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb3a56cb32a47499d52173bb3981c59 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ca063a19a044e6c8466598f44316ff5) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.522441 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe6fb5025f5847f8bb3617c22bdf8c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.522516 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ca063a19a044e6c8466598f44316ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.523227 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 2ca063a19a044e6c8466598f44316ff5 (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.523298 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ca063a19a044e6c8466598f44316ff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.522471 27844 ts_tablet_manager.cc:1888] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.523543 27844 log.cc:1196] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d212d041e0854d1b9cc872286ecbd6ea
I20240503 02:09:07.523581 27843 ts_tablet_manager.cc:1875] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.523741 27844 ts_tablet_manager.cc:1909] T d212d041e0854d1b9cc872286ecbd6ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.523922 27844 tablet_replica.cc:330] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.523993 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.524015 27844 raft_consensus.cc:2229] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.524045 27844 raft_consensus.cc:2258] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.524230 27844 ts_tablet_manager.cc:1875] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.524294 27839 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.524327 27839 ts_tablet_manager.cc:1367] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:07.524439 27839 raft_consensus.cc:348] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.524487 27839 raft_consensus.cc:374] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.524503 27839 raft_consensus.cc:729] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.524545 27839 consensus_queue.cc:260] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.524645 27839 ts_tablet_manager.cc:1398] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.524693 27839 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.525000 27845 ts_tablet_manager.cc:1888] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.525017 27839 tablet_bootstrap.cc:654] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.525041 27845 log.cc:1196] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/957e479707bc4c25ae0284bd02031caa
I20240503 02:09:07.525179 27843 ts_tablet_manager.cc:1888] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.525209 27843 log.cc:1196] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fcb3a56cb32a47499d52173bb3981c59
I20240503 02:09:07.525480 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.525532 27839 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
W20240503 02:09:07.525557 27122 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet fe6fb5025f5847f8bb3617c22bdf8c66 consensus state has no leader
I20240503 02:09:07.525564 27845 ts_tablet_manager.cc:1909] T 957e479707bc4c25ae0284bd02031caa P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.525568 27839 ts_tablet_manager.cc:1367] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.525751 27844 ts_tablet_manager.cc:1888] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.525791 27844 log.cc:1196] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fcb3a56cb32a47499d52173bb3981c59
I20240503 02:09:07.525884 27843 ts_tablet_manager.cc:1909] T fcb3a56cb32a47499d52173bb3981c59 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.525888 27839 raft_consensus.cc:348] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.525929 27844 ts_tablet_manager.cc:1909] T fcb3a56cb32a47499d52173bb3981c59 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.525943 27839 raft_consensus.cc:374] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.525961 27839 raft_consensus.cc:729] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.526006 27839 consensus_queue.cc:260] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.526099 27839 ts_tablet_manager.cc:1398] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.526343 27838 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.526386 27838 ts_tablet_manager.cc:1367] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:07.526495 27838 raft_consensus.cc:348] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.526552 27838 raft_consensus.cc:374] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.526574 27838 raft_consensus.cc:729] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.526611 27838 consensus_queue.cc:260] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.526638 27845 tablet_replica.cc:330] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.526676 27845 raft_consensus.cc:2229] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.526688 27838 ts_tablet_manager.cc:1398] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.526746 27845 pending_rounds.cc:62] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:07.526732 27838 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.526767 27845 pending_rounds.cc:68] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419996083892224 op_type: NO_OP noop_request { }
I20240503 02:09:07.526806 27845 raft_consensus.cc:2866] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.526830 27845 raft_consensus.cc:2258] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.527168 27838 tablet_bootstrap.cc:654] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.527436 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fcb3a56cb32a47499d52173bb3981c59 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:07.527534 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fcb3a56cb32a47499d52173bb3981c59 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:07.527604 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 957e479707bc4c25ae0284bd02031caa (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.527623 27838 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.527660 27838 ts_tablet_manager.cc:1367] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.527746 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.527781 27838 raft_consensus.cc:348] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.527827 27838 raft_consensus.cc:374] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.527863 27838 raft_consensus.cc:729] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.527900 27838 consensus_queue.cc:260] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.527971 27845 ts_tablet_manager.cc:1875] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.527994 27838 ts_tablet_manager.cc:1398] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.528288 27837 tablet_bootstrap.cc:492] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.528324 27837 ts_tablet_manager.cc:1367] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:07.528439 27837 raft_consensus.cc:348] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.528487 27837 raft_consensus.cc:374] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.528503 27837 raft_consensus.cc:729] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.528545 27837 consensus_queue.cc:260] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.528627 27837 ts_tablet_manager.cc:1398] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.528671 27837 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.528970 27845 ts_tablet_manager.cc:1888] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.529008 27845 log.cc:1196] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fa37b879af0a4b26982515c82198d6c6
I20240503 02:09:07.529043 27837 tablet_bootstrap.cc:654] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.529170 27845 ts_tablet_manager.cc:1909] T fa37b879af0a4b26982515c82198d6c6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.529618 27837 tablet_bootstrap.cc:492] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.529660 27837 ts_tablet_manager.cc:1367] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.529706 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.529721 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.529811 27837 raft_consensus.cc:348] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.529862 27837 raft_consensus.cc:374] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.529879 27837 raft_consensus.cc:729] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.529927 27837 consensus_queue.cc:260] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.530227 27837 ts_tablet_manager.cc:1398] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.530294 27837 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.530573 27845 tablet_replica.cc:330] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.530668 27837 tablet_bootstrap.cc:654] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.530701 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fa37b879af0a4b26982515c82198d6c6 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.530742 27845 raft_consensus.cc:2229] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.530859 27845 raft_consensus.cc:2258] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.531122 27845 ts_tablet_manager.cc:1875] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.531134 27837 tablet_bootstrap.cc:492] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.531167 27837 ts_tablet_manager.cc:1367] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.531291 27837 raft_consensus.cc:348] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.531337 27837 raft_consensus.cc:374] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.531353 27837 raft_consensus.cc:729] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.531387 27837 consensus_queue.cc:260] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.531466 27837 ts_tablet_manager.cc:1398] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.531512 27837 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.531785 27837 tablet_bootstrap.cc:654] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.532070 27845 ts_tablet_manager.cc:1888] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.532167 27845 log.cc:1196] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d212d041e0854d1b9cc872286ecbd6ea
I20240503 02:09:07.532229 27837 tablet_bootstrap.cc:492] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.532277 27837 ts_tablet_manager.cc:1367] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.532403 27837 raft_consensus.cc:348] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.532461 27837 raft_consensus.cc:374] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.532492 27837 raft_consensus.cc:729] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.532516 27837 consensus_queue.cc:260] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.532596 27837 ts_tablet_manager.cc:1398] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.532665 27845 ts_tablet_manager.cc:1909] T d212d041e0854d1b9cc872286ecbd6ea P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.532779 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.532831 27846 raft_consensus.cc:482] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.532876 27846 raft_consensus.cc:504] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.532905 27845 tablet_replica.cc:330] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.532940 27845 raft_consensus.cc:2229] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.532969 27845 pending_rounds.cc:62] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:07.532977 27846 leader_election.cc:288] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.532985 27845 pending_rounds.cc:68] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419996238901248 op_type: NO_OP noop_request { }
I20240503 02:09:07.533036 27845 raft_consensus.cc:2866] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.533053 27845 raft_consensus.cc:2258] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.533095 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d212d041e0854d1b9cc872286ecbd6ea (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.533257 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe6fb5025f5847f8bb3617c22bdf8c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.533314 27723 raft_consensus.cc:2454] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.533308 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe6fb5025f5847f8bb3617c22bdf8c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.533437 27586 raft_consensus.cc:2454] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.533445 27388 leader_election.cc:302] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.533504 27846 raft_consensus.cc:2781] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.533522 27845 ts_tablet_manager.cc:1875] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.533538 27846 raft_consensus.cc:482] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.533556 27846 raft_consensus.cc:3037] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.534092 27846 raft_consensus.cc:504] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.534205 27846 leader_election.cc:288] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.534373 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe6fb5025f5847f8bb3617c22bdf8c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.534430 27586 raft_consensus.cc:3037] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.534446 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe6fb5025f5847f8bb3617c22bdf8c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.534498 27723 raft_consensus.cc:3037] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.534570 27845 ts_tablet_manager.cc:1888] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.534605 27845 log.cc:1196] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fcb3a56cb32a47499d52173bb3981c59
I20240503 02:09:07.534771 27845 ts_tablet_manager.cc:1909] T fcb3a56cb32a47499d52173bb3981c59 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.534868 27586 raft_consensus.cc:2454] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.535024 27723 raft_consensus.cc:2454] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.535109 27386 leader_election.cc:302] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.535133 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fcb3a56cb32a47499d52173bb3981c59 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:07.535185 27846 raft_consensus.cc:2781] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.535228 27846 raft_consensus.cc:686] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.535259 27846 consensus_queue.cc:237] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.535885 27122 catalog_manager.cc:5484] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.535964 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.541091 27846 raft_consensus.cc:482] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.541137 27846 raft_consensus.cc:504] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.541229 27846 leader_election.cc:288] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.541379 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ca063a19a044e6c8466598f44316ff5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.541384 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ca063a19a044e6c8466598f44316ff5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.541438 27723 raft_consensus.cc:2454] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.541437 27586 raft_consensus.cc:2454] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.541563 27388 leader_election.cc:302] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.541671 27846 raft_consensus.cc:2781] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.541707 27846 raft_consensus.cc:482] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.541718 27846 raft_consensus.cc:3037] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.542153 27846 raft_consensus.cc:504] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.542250 27846 leader_election.cc:288] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.542366 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ca063a19a044e6c8466598f44316ff5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.542430 27723 raft_consensus.cc:3037] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.542456 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ca063a19a044e6c8466598f44316ff5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.542518 27586 raft_consensus.cc:3037] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.542845 27723 raft_consensus.cc:2454] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.542940 27586 raft_consensus.cc:2454] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.542980 27388 leader_election.cc:302] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.543056 27846 raft_consensus.cc:2781] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.543107 27846 raft_consensus.cc:686] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.543139 27846 consensus_queue.cc:237] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.543622 27122 catalog_manager.cc:5484] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.552618 27884 raft_consensus.cc:482] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.552690 27884 raft_consensus.cc:504] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.552798 27884 leader_election.cc:288] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.552937 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff52b758952450dbc6eb3162457000f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.552979 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff52b758952450dbc6eb3162457000f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.553004 27453 raft_consensus.cc:2454] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.553025 27723 raft_consensus.cc:2454] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.553125 27524 leader_election.cc:302] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.553195 27884 raft_consensus.cc:2781] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.553223 27884 raft_consensus.cc:482] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.553249 27884 raft_consensus.cc:3037] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.553714 27884 raft_consensus.cc:504] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.553814 27884 leader_election.cc:288] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.553921 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff52b758952450dbc6eb3162457000f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.553943 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff52b758952450dbc6eb3162457000f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.553979 27723 raft_consensus.cc:3037] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.553989 27453 raft_consensus.cc:3037] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.554332 27723 raft_consensus.cc:2454] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.554402 27453 raft_consensus.cc:2454] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.554499 27523 leader_election.cc:302] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.554594 27884 raft_consensus.cc:2781] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.554644 27884 raft_consensus.cc:686] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.554687 27884 consensus_queue.cc:237] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.555236 27122 catalog_manager.cc:5484] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.555903 27846 raft_consensus.cc:482] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.555948 27846 raft_consensus.cc:504] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.556056 27846 leader_election.cc:288] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.556268 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4210a30264df45c89eec0a6bebe3b46c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.556290 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4210a30264df45c89eec0a6bebe3b46c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.556332 27586 raft_consensus.cc:2454] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.556344 27723 raft_consensus.cc:2454] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.556445 27386 leader_election.cc:302] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.556511 27846 raft_consensus.cc:2781] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.556552 27846 raft_consensus.cc:482] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.556578 27846 raft_consensus.cc:3037] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.556921 27846 raft_consensus.cc:504] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.557022 27846 leader_election.cc:288] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.557147 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4210a30264df45c89eec0a6bebe3b46c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.557205 27586 raft_consensus.cc:3037] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.557190 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4210a30264df45c89eec0a6bebe3b46c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.557237 27723 raft_consensus.cc:3037] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.557734 27723 raft_consensus.cc:2454] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.557734 27586 raft_consensus.cc:2454] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.557902 27386 leader_election.cc:302] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.557993 27846 raft_consensus.cc:2781] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.558050 27846 raft_consensus.cc:686] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.558092 27846 consensus_queue.cc:237] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.558522 27122 catalog_manager.cc:5484] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.558611 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]). Expected version 2 got 0
I20240503 02:09:07.574494 27723 raft_consensus.cc:1261] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.574506 27586 raft_consensus.cc:1261] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.574697 27846 consensus_queue.cc:1035] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.574795 27852 consensus_queue.cc:1035] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.575532 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.575963 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.576076 27912 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.576259 27917 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.576591 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.578235 27904 raft_consensus.cc:482] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.578286 27904 raft_consensus.cc:504] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.578394 27904 leader_election.cc:288] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.578485 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b7b16f82564b1ba88df6c72e41eb62" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.578497 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b7b16f82564b1ba88df6c72e41eb62" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.578543 27453 raft_consensus.cc:2454] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.578557 27586 raft_consensus.cc:2454] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.578651 27659 leader_election.cc:302] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.578719 27983 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:07.578959 27904 raft_consensus.cc:2781] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.579000 27904 raft_consensus.cc:482] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.579016 27904 raft_consensus.cc:3037] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.579526 27904 raft_consensus.cc:504] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.579649 27904 leader_election.cc:288] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.579731 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b7b16f82564b1ba88df6c72e41eb62" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.579774 27453 raft_consensus.cc:3037] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.579759 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97b7b16f82564b1ba88df6c72e41eb62" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.579809 27586 raft_consensus.cc:3037] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.580044 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_3e3b0d9c62d94690813b02f4646a0adf [id=ffba5f39864e4bc29d61fd32c9f57d31]). Expected version 2 got 0
I20240503 02:09:07.580212 27586 raft_consensus.cc:2454] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.580309 27453 raft_consensus.cc:2454] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.580513 27659 leader_election.cc:302] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.580579 27904 raft_consensus.cc:2781] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.580615 27904 raft_consensus.cc:686] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.580657 27904 consensus_queue.cc:237] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.581171 27122 catalog_manager.cc:5484] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.581267 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.614954 27853 raft_consensus.cc:482] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.615012 27853 raft_consensus.cc:504] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.615128 27853 leader_election.cc:288] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:07.615298 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e82752def4a31b3d8bec18a478273" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.615366 27586 raft_consensus.cc:2454] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.615334 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e82752def4a31b3d8bec18a478273" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.615398 27453 raft_consensus.cc:2454] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.615545 27659 leader_election.cc:302] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.615621 27853 raft_consensus.cc:2781] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.615651 27853 raft_consensus.cc:482] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.615675 27853 raft_consensus.cc:3037] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.616096 27853 raft_consensus.cc:504] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.616192 27853 leader_election.cc:288] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:07.616310 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e82752def4a31b3d8bec18a478273" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.616323 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e82752def4a31b3d8bec18a478273" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.616364 27586 raft_consensus.cc:3037] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.616379 27453 raft_consensus.cc:3037] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.616751 27586 raft_consensus.cc:2454] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.616791 27453 raft_consensus.cc:2454] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.616920 27659 leader_election.cc:302] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.617010 27853 raft_consensus.cc:2781] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.617058 27853 raft_consensus.cc:686] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.617102 27853 consensus_queue.cc:237] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.617591 27122 catalog_manager.cc:5484] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.618997 27453 raft_consensus.cc:1261] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.618997 27586 raft_consensus.cc:1261] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.619189 27904 consensus_queue.cc:1035] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.619274 27853 consensus_queue.cc:1035] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.620066 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.620208 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]). Expected version 1 got 0
I20240503 02:09:07.620368 27983 tablet.cc:1722] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.620820 27912 tablet.cc:1722] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.622078 27123 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea] alter complete (version 1)
I20240503 02:09:07.622154 27917 tablet.cc:1722] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.639078 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_2e984ea13c8c4a0d884abb238e6c64ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.639173 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e984ea13c8c4a0d884abb238e6c64ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.640866 27429 tablet_service.cc:1467] Processing CreateTablet for tablet db95b9847807414aa63e686837a0c9c1 (DEFAULT_TABLE table=default.Table_2e984ea13c8c4a0d884abb238e6c64ae [id=5553fa58eac145c885b8f2325c4ef52d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.640938 27568 tablet_service.cc:1467] Processing CreateTablet for tablet db95b9847807414aa63e686837a0c9c1 (DEFAULT_TABLE table=default.Table_2e984ea13c8c4a0d884abb238e6c64ae [id=5553fa58eac145c885b8f2325c4ef52d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.640970 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db95b9847807414aa63e686837a0c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.641043 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db95b9847807414aa63e686837a0c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.641269 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_ed1bf73042484037a064b0da2ee4ef69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.641386 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed1bf73042484037a064b0da2ee4ef69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.641988 27839 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.642043 27837 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.642134 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_597335db18b74d859f4f7a0fcd41def7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.642235 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_597335db18b74d859f4f7a0fcd41def7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.642241 27839 tablet_bootstrap.cc:654] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.642370 27837 tablet_bootstrap.cc:654] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.642518 27702 tablet_service.cc:1467] Processing CreateTablet for tablet db95b9847807414aa63e686837a0c9c1 (DEFAULT_TABLE table=default.Table_2e984ea13c8c4a0d884abb238e6c64ae [id=5553fa58eac145c885b8f2325c4ef52d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.642614 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db95b9847807414aa63e686837a0c9c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.642671 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_8918d4d6716f4974a4f660ca3eb42f17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.642758 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8918d4d6716f4974a4f660ca3eb42f17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.642889 27130 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_4653432f1d9041cc83d5f0beb9ac8b6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.642949 27839 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
W20240503 02:09:07.642967 27130 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.642982 27839 ts_tablet_manager.cc:1367] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.643108 27839 raft_consensus.cc:348] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.643163 27839 raft_consensus.cc:374] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.643182 27839 raft_consensus.cc:729] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.643224 27839 consensus_queue.cc:260] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.643309 27839 ts_tablet_manager.cc:1398] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.643453 27837 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.643481 27837 ts_tablet_manager.cc:1367] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.643568 27837 raft_consensus.cc:348] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.643602 27837 raft_consensus.cc:374] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.643612 27837 raft_consensus.cc:729] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.643646 27837 consensus_queue.cc:260] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.643673 27838 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.643723 27837 ts_tablet_manager.cc:1398] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.644018 27838 tablet_bootstrap.cc:654] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.645004 27838 tablet_bootstrap.cc:492] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.645049 27838 ts_tablet_manager.cc:1367] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.645166 27838 raft_consensus.cc:348] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.645217 27838 raft_consensus.cc:374] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.645234 27838 raft_consensus.cc:729] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.645267 27838 consensus_queue.cc:260] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.645363 27838 ts_tablet_manager.cc:1398] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.646260 27130 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_16a939c3436b4c0fabceb3838811d215"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.646361 27130 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16a939c3436b4c0fabceb3838811d215 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.646271 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_059540465a8f493596bbda1239ae26b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.646637 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_059540465a8f493596bbda1239ae26b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.647193 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 960d68053f9642ff82c9c30983973e93 (DEFAULT_TABLE table=default.Table_ed1bf73042484037a064b0da2ee4ef69 [id=2a045f55a9c84ca88188f62b60a8cb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.647293 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960d68053f9642ff82c9c30983973e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.647338 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 960d68053f9642ff82c9c30983973e93 (DEFAULT_TABLE table=default.Table_ed1bf73042484037a064b0da2ee4ef69 [id=2a045f55a9c84ca88188f62b60a8cb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.647426 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960d68053f9642ff82c9c30983973e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.647572 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.647641 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c66d743724b81ab178b9bf0405b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.648182 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.648244 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c66d743724b81ab178b9bf0405b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.648521 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 960d68053f9642ff82c9c30983973e93 (DEFAULT_TABLE table=default.Table_ed1bf73042484037a064b0da2ee4ef69 [id=2a045f55a9c84ca88188f62b60a8cb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.648600 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 960d68053f9642ff82c9c30983973e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.647212 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.649022 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c66d743724b81ab178b9bf0405b90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.649505 27839 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.649758 27839 tablet_bootstrap.cc:654] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.650031 27837 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.650111 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_ef7d98c2a6d34b4a9416df1897d18064" } new_table_name: "default.Table_5cfc506f31b1478d8d585da047adff86" modify_external_catalogs: true
I20240503 02:09:07.650130 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_99f1ffda145649899d3aa39d2147bdd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.650203 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99f1ffda145649899d3aa39d2147bdd4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.650321 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_90f372f788f64dd0af4c4b9718ade63a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.650388 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90f372f788f64dd0af4c4b9718ade63a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.650441 27837 tablet_bootstrap.cc:654] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.650734 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_ed1bf73042484037a064b0da2ee4ef69" } new_table_name: "default.Table_38ee0012453d4e54b8c173c8129d3b74" modify_external_catalogs: true
I20240503 02:09:07.650799 27838 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.650913 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_7a3d1471909649efb6bf2ddb710a07e4" } new_table_name: "default.Table_b286e68e663d42dfa92e9bb689b0fb4b" modify_external_catalogs: true
I20240503 02:09:07.651131 27838 tablet_bootstrap.cc:654] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.651307 27839 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.651348 27839 ts_tablet_manager.cc:1367] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.651458 27839 raft_consensus.cc:348] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.651520 27839 raft_consensus.cc:374] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.651538 27839 raft_consensus.cc:729] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.651572 27839 consensus_queue.cc:260] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.651649 27839 ts_tablet_manager.cc:1398] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.651657 27838 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.651693 27838 ts_tablet_manager.cc:1367] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.651711 27839 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.651810 27838 raft_consensus.cc:348] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.651867 27838 raft_consensus.cc:374] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.651885 27838 raft_consensus.cc:729] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.651916 27838 consensus_queue.cc:260] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.651985 27838 ts_tablet_manager.cc:1398] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.652040 27838 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.652043 27839 tablet_bootstrap.cc:654] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.652407 27838 tablet_bootstrap.cc:654] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.652844 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 51172f389028420fb0690adea9673429 (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.652925 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51172f389028420fb0690adea9673429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.652962 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (DEFAULT_TABLE table=default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.653048 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.653776 27453 raft_consensus.cc:1261] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.654053 27900 consensus_queue.cc:1035] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.654129 27586 raft_consensus.cc:1261] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:09:07.654173 27127 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 960d68053f9642ff82c9c30983973e93 consensus state has no leader
I20240503 02:09:07.654385 27838 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.654436 27838 ts_tablet_manager.cc:1367] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.654366 27900 consensus_queue.cc:1035] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.654557 27838 raft_consensus.cc:348] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.654609 27838 raft_consensus.cc:374] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.654628 27838 raft_consensus.cc:729] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.654661 27838 consensus_queue.cc:260] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.654701 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 51172f389028420fb0690adea9673429 (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.654805 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51172f389028420fb0690adea9673429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.655395 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (DEFAULT_TABLE table=default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.655476 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.654732 27838 ts_tablet_manager.cc:1398] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.655848 27838 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.656183 27838 tablet_bootstrap.cc:654] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.656555 27568 tablet_service.cc:1467] Processing CreateTablet for tablet 291c3bbf65b5464cbf8521617fba6365 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.656605 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 (DEFAULT_TABLE table=default.Table_90f372f788f64dd0af4c4b9718ade63a [id=3383f9af04da490a8cb6f949d3d92644]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.656636 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291c3bbf65b5464cbf8521617fba6365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.656690 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e419af3f80e40b8a1acdfb71550ee91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.657348 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.657478 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:07.657483 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 203665395f3745f196fb393dd82b8fc9 (DEFAULT_TABLE table=default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657560 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203665395f3745f196fb393dd82b8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.657716 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (DEFAULT_TABLE table=default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657773 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 442400993f8e43b79e429758d1f3c72f (DEFAULT_TABLE table=default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657781 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 203665395f3745f196fb393dd82b8fc9 (DEFAULT_TABLE table=default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657833 27428 tablet_service.cc:1467] Processing CreateTablet for tablet 291c3bbf65b5464cbf8521617fba6365 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657850 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442400993f8e43b79e429758d1f3c72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.657902 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 (DEFAULT_TABLE table=default.Table_90f372f788f64dd0af4c4b9718ade63a [id=3383f9af04da490a8cb6f949d3d92644]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657915 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 442400993f8e43b79e429758d1f3c72f (DEFAULT_TABLE table=default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.657790 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.658126 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291c3bbf65b5464cbf8521617fba6365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.658605 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 51172f389028420fb0690adea9673429 (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.658674 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51172f389028420fb0690adea9673429. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.658927 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e419af3f80e40b8a1acdfb71550ee91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659094 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442400993f8e43b79e429758d1f3c72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659231 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 203665395f3745f196fb393dd82b8fc9 (DEFAULT_TABLE table=default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.659309 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203665395f3745f196fb393dd82b8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659454 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 291c3bbf65b5464cbf8521617fba6365 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.659523 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291c3bbf65b5464cbf8521617fba6365. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659617 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 (DEFAULT_TABLE table=default.Table_90f372f788f64dd0af4c4b9718ade63a [id=3383f9af04da490a8cb6f949d3d92644]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.659698 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e419af3f80e40b8a1acdfb71550ee91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659780 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 442400993f8e43b79e429758d1f3c72f (DEFAULT_TABLE table=default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.659838 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 442400993f8e43b79e429758d1f3c72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.659909 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 203665395f3745f196fb393dd82b8fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.660768 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_3f10232e10f94de69e51315eb0cd121b" } new_table_name: "default.Table_29c62440de5041adb7dc3747d41924ee" modify_external_catalogs: true
I20240503 02:09:07.660797 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_3e3b0d9c62d94690813b02f4646a0adf" } new_table_name: "default.Table_e4374c6711ad425faf97b03b048df0ec" modify_external_catalogs: true
I20240503 02:09:07.661128 27125 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.661293 27837 tablet_bootstrap.cc:492] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.661299 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.661351 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:07.661356 27837 ts_tablet_manager.cc:1367] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:07.661481 27837 raft_consensus.cc:348] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.661540 27837 raft_consensus.cc:374] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.661568 27837 raft_consensus.cc:729] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.661609 27837 consensus_queue.cc:260] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.661696 27837 ts_tablet_manager.cc:1398] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.661778 27837 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.662096 27837 tablet_bootstrap.cc:654] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.662169 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.662760 27723 raft_consensus.cc:1261] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.662954 27837 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.662961 28007 consensus_queue.cc:1035] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.663005 27837 ts_tablet_manager.cc:1367] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.663295 27837 raft_consensus.cc:348] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.663347 27837 raft_consensus.cc:374] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.663364 27837 raft_consensus.cc:729] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.663399 27837 consensus_queue.cc:260] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.663475 27837 ts_tablet_manager.cc:1398] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.663528 27837 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.663837 27837 tablet_bootstrap.cc:654] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.663901 27587 raft_consensus.cc:1261] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.663944 27912 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:07.664054 27452 raft_consensus.cc:1261] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.664060 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9ff52b758952450dbc6eb3162457000f (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.664343 27839 tablet_bootstrap.cc:492] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.664383 27839 ts_tablet_manager.cc:1367] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:07.664498 27839 raft_consensus.cc:348] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.664543 27839 raft_consensus.cc:374] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.664561 27839 raft_consensus.cc:729] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.664597 27839 consensus_queue.cc:260] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.664675 27839 ts_tablet_manager.cc:1398] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.664845 28007 consensus_queue.cc:1035] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.665159 27904 consensus_queue.cc:1035] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.665838 27839 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.665896 27983 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:07.665943 27127 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_d313bd2d1c434bbc8d5d364828596066" } modify_external_catalogs: true
I20240503 02:09:07.665997 27127 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_d313bd2d1c434bbc8d5d364828596066" } modify_external_catalogs: true
I20240503 02:09:07.666150 27839 tablet_bootstrap.cc:654] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.666663 27122 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_e4374c6711ad425faf97b03b048df0ec [id=ffba5f39864e4bc29d61fd32c9f57d31] alter complete (version 3)
I20240503 02:09:07.667205 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 4210a30264df45c89eec0a6bebe3b46c
I20240503 02:09:07.667506 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4210a30264df45c89eec0a6bebe3b46c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.667526 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet 4210a30264df45c89eec0a6bebe3b46c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.667598 27845 tablet_replica.cc:330] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.667608 27843 tablet_replica.cc:330] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.667652 27845 raft_consensus.cc:2229] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.667657 27843 raft_consensus.cc:2229] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.667686 27843 raft_consensus.cc:2258] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.667692 27845 pending_rounds.cc:62] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:07.667708 27845 pending_rounds.cc:68] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419996398145536 op_type: NO_OP noop_request { }
I20240503 02:09:07.667740 27845 raft_consensus.cc:2866] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.667768 27845 raft_consensus.cc:2258] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.667832 27843 ts_tablet_manager.cc:1875] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.667954 27845 ts_tablet_manager.cc:1875] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.668097 27127 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_2691a07efb4c43e698e53947e6b90445" } modify_external_catalogs: true
I20240503 02:09:07.668149 27127 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_2691a07efb4c43e698e53947e6b90445" } modify_external_catalogs: true
I20240503 02:09:07.668932 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_786b4055ce5140d08d17981aef2d77b2" } modify_external_catalogs: true
I20240503 02:09:07.668962 27845 ts_tablet_manager.cc:1888] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.668984 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_786b4055ce5140d08d17981aef2d77b2" } modify_external_catalogs: true
I20240503 02:09:07.668995 27845 log.cc:1196] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4210a30264df45c89eec0a6bebe3b46c
I20240503 02:09:07.669118 27845 ts_tablet_manager.cc:1909] T 4210a30264df45c89eec0a6bebe3b46c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.669306 27843 ts_tablet_manager.cc:1888] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.669376 27843 log.cc:1196] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4210a30264df45c89eec0a6bebe3b46c
I20240503 02:09:07.669515 27843 ts_tablet_manager.cc:1909] T 4210a30264df45c89eec0a6bebe3b46c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.669507 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.669749 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.669807 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet faf4927166d74721a6d1890adadd3019
I20240503 02:09:07.670047 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet faf4927166d74721a6d1890adadd3019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.670064 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet faf4927166d74721a6d1890adadd3019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.670148 27845 tablet_replica.cc:330] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.670208 27845 raft_consensus.cc:2229] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.670238 27845 raft_consensus.cc:2258] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.670302 27843 tablet_replica.cc:330] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.670357 27843 raft_consensus.cc:2229] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.670415 27843 raft_consensus.cc:2258] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.670565 27845 ts_tablet_manager.cc:1875] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.670650 27843 ts_tablet_manager.cc:1875] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.671116 27904 raft_consensus.cc:482] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.671159 27904 raft_consensus.cc:504] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.671253 27904 leader_election.cc:288] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.671478 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 97b7b16f82564b1ba88df6c72e41eb62
I20240503 02:09:07.671563 27845 ts_tablet_manager.cc:1888] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.671607 27845 log.cc:1196] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/faf4927166d74721a6d1890adadd3019
I20240503 02:09:07.671660 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.671775 27845 ts_tablet_manager.cc:1909] T faf4927166d74721a6d1890adadd3019 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.671772 27843 ts_tablet_manager.cc:1888] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.671818 27843 log.cc:1196] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/faf4927166d74721a6d1890adadd3019
I20240503 02:09:07.671978 27845 tablet_replica.cc:330] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.671998 27843 ts_tablet_manager.cc:1909] T faf4927166d74721a6d1890adadd3019 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.672026 27845 raft_consensus.cc:2229] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.672055 27845 raft_consensus.cc:2258] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.672039 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet faf4927166d74721a6d1890adadd3019 (table default.Table_2691a07efb4c43e698e53947e6b90445 [id=10fd8d0ff7104ffe99d73720ff9a5f76]) successfully deleted
I20240503 02:09:07.672184 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.672209 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet faf4927166d74721a6d1890adadd3019 (table default.Table_2691a07efb4c43e698e53947e6b90445 [id=10fd8d0ff7104ffe99d73720ff9a5f76]) successfully deleted
I20240503 02:09:07.672257 27843 tablet_replica.cc:330] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.672312 27843 raft_consensus.cc:2229] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.672353 27843 raft_consensus.cc:2258] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.672441 27845 ts_tablet_manager.cc:1875] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.672571 27843 ts_tablet_manager.cc:1875] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.673411 27845 ts_tablet_manager.cc:1888] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.673470 27845 log.cc:1196] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/97b7b16f82564b1ba88df6c72e41eb62
I20240503 02:09:07.673514 27843 ts_tablet_manager.cc:1888] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.673563 27843 log.cc:1196] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/97b7b16f82564b1ba88df6c72e41eb62
I20240503 02:09:07.673635 27845 ts_tablet_manager.cc:1909] T 97b7b16f82564b1ba88df6c72e41eb62 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.673748 27843 ts_tablet_manager.cc:1909] T 97b7b16f82564b1ba88df6c72e41eb62 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.673902 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]) successfully deleted
I20240503 02:09:07.674005 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]) successfully deleted
I20240503 02:09:07.675266 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_f44cea6f7c3d4047bbabe392bba65b57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.675350 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f44cea6f7c3d4047bbabe392bba65b57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.676821 27127 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc" } modify_external_catalogs: true
I20240503 02:09:07.676890 27127 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc" } modify_external_catalogs: true
I20240503 02:09:07.677807 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 506cf6ad0d2645a491252b525117e9ad
I20240503 02:09:07.678020 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 506cf6ad0d2645a491252b525117e9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.678028 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet 506cf6ad0d2645a491252b525117e9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.678088 27845 tablet_replica.cc:330] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.678144 27845 raft_consensus.cc:2229] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.678180 27845 raft_consensus.cc:2258] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.678426 27845 ts_tablet_manager.cc:1875] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.679504 27843 tablet_replica.cc:330] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.679558 27843 raft_consensus.cc:2229] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.679606 27843 raft_consensus.cc:2258] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.679597 27845 ts_tablet_manager.cc:1888] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.679641 27845 log.cc:1196] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/506cf6ad0d2645a491252b525117e9ad
I20240503 02:09:07.679805 27845 ts_tablet_manager.cc:1909] T 506cf6ad0d2645a491252b525117e9ad P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.680184 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]) successfully deleted
I20240503 02:09:07.680761 28007 raft_consensus.cc:482] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.680817 28007 raft_consensus.cc:504] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.680917 28007 leader_election.cc:288] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.680976 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_9017d82aa4984e29a21642db1dd0c595" } modify_external_catalogs: true
I20240503 02:09:07.681059 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_9017d82aa4984e29a21642db1dd0c595" } modify_external_catalogs: true
I20240503 02:09:07.681150 27843 ts_tablet_manager.cc:1875] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.682199 27843 ts_tablet_manager.cc:1888] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.682281 27843 log.cc:1196] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/506cf6ad0d2645a491252b525117e9ad
I20240503 02:09:07.682502 27843 ts_tablet_manager.cc:1909] T 506cf6ad0d2645a491252b525117e9ad P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.683843 27838 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.683902 27838 ts_tablet_manager.cc:1367] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20240503 02:09:07.684012 27838 raft_consensus.cc:348] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.684060 27838 raft_consensus.cc:374] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.684077 27838 raft_consensus.cc:729] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.684113 27838 consensus_queue.cc:260] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.684127 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db95b9847807414aa63e686837a0c9c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.684185 27838 ts_tablet_manager.cc:1398] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.684190 27452 raft_consensus.cc:2454] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.684253 27838 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.684262 27453 raft_consensus.cc:1261] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.684515 27659 leader_election.cc:302] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.684581 27838 tablet_bootstrap.cc:654] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.684701 27894 consensus_queue.cc:1035] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.684796 27893 tablet.cc:1722] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.684705 27903 raft_consensus.cc:2781] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.684907 27903 raft_consensus.cc:482] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.684924 27903 raft_consensus.cc:3037] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.685392 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.685446 27722 raft_consensus.cc:2454] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.685540 27903 raft_consensus.cc:504] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.685629 27903 leader_election.cc:288] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.685724 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]) successfully deleted
I20240503 02:09:07.685636 27838 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.685874 27838 ts_tablet_manager.cc:1367] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.685982 27838 raft_consensus.cc:348] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.686017 27838 raft_consensus.cc:374] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.686028 27838 raft_consensus.cc:729] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.686059 27838 consensus_queue.cc:260] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.686146 27838 ts_tablet_manager.cc:1398] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.686179 27838 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.686378 27846 raft_consensus.cc:482] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.686424 27846 raft_consensus.cc:504] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.686507 27846 leader_election.cc:288] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.686544 27838 tablet_bootstrap.cc:654] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.686607 27523 leader_election.cc:302] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.686714 27884 raft_consensus.cc:2781] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.686751 27884 raft_consensus.cc:482] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.686774 27884 raft_consensus.cc:3037] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.686744 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeba88b72524e9cb77720cb7ca13597" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.686803 27723 raft_consensus.cc:2454] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.686875 27917 tablet.cc:1722] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.687042 27848 raft_consensus.cc:482] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.687055 27838 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.687048 27388 leader_election.cc:302] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.687088 27838 ts_tablet_manager.cc:1367] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.687074 27848 raft_consensus.cc:504] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.687141 27848 leader_election.cc:288] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:07.687139 27846 raft_consensus.cc:2781] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.687168 27846 raft_consensus.cc:482] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.687186 27846 raft_consensus.cc:3037] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.687204 27838 raft_consensus.cc:348] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.687249 27838 raft_consensus.cc:374] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.687266 27838 raft_consensus.cc:729] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.687299 27838 consensus_queue.cc:260] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.687379 27838 ts_tablet_manager.cc:1398] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.687418 27838 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.687650 27893 tablet.cc:1722] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.687731 27838 tablet_bootstrap.cc:654] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.687777 27846 raft_consensus.cc:504] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.687875 27846 leader_election.cc:288] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.688057 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db95b9847807414aa63e686837a0c9c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.688114 27451 raft_consensus.cc:3037] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.688313 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960d68053f9642ff82c9c30983973e93" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.688380 27722 raft_consensus.cc:2454] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.688477 27125 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9614b6572dfd443387ee3e4d676e8766 (table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc]). Expected version 1 got 0
I20240503 02:09:07.688504 27125 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:07.688498 27838 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.688539 27838 ts_tablet_manager.cc:1367] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.688611 27912 tablet.cc:1722] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.688647 27838 raft_consensus.cc:348] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.688699 27838 raft_consensus.cc:374] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.688696 27451 raft_consensus.cc:2454] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.688716 27838 raft_consensus.cc:729] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.688793 27838 consensus_queue.cc:260] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.688989 27523 leader_election.cc:302] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.688910 27838 ts_tablet_manager.cc:1398] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.689021 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeba88b72524e9cb77720cb7ca13597" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.689052 27838 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.689065 27848 raft_consensus.cc:2781] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.689087 27848 raft_consensus.cc:482] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.689074 27722 raft_consensus.cc:3037] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.689106 27848 raft_consensus.cc:3037] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.689357 27659 leader_election.cc:302] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.689436 27838 tablet_bootstrap.cc:654] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.689616 27837 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.689652 27837 ts_tablet_manager.cc:1367] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.001s
I20240503 02:09:07.689741 27722 raft_consensus.cc:2454] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.689853 27838 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.689886 27838 ts_tablet_manager.cc:1367] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.690011 27838 raft_consensus.cc:348] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.690073 27838 raft_consensus.cc:374] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.689772 27837 raft_consensus.cc:348] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.690097 27838 raft_consensus.cc:729] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.690117 27837 raft_consensus.cc:374] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.690096 27848 raft_consensus.cc:504] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.690122 27838 consensus_queue.cc:260] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.690147 27837 raft_consensus.cc:729] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.690191 27838 ts_tablet_manager.cc:1398] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.690184 27848 leader_election.cc:288] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:07.690256 27838 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.690443 27894 raft_consensus.cc:2781] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.690492 27894 raft_consensus.cc:686] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.690469 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeba88b72524e9cb77720cb7ca13597" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.690532 27586 raft_consensus.cc:2454] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.690526 27894 consensus_queue.cc:237] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.690582 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet faf4927166d74721a6d1890adadd3019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.690649 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 506cf6ad0d2645a491252b525117e9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.690670 27838 tablet_bootstrap.cc:654] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.690789 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbeba88b72524e9cb77720cb7ca13597" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.690862 27587 raft_consensus.cc:3037] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.690913 27844 tablet_replica.cc:330] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.690954 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 97b7b16f82564b1ba88df6c72e41eb62 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.690972 27844 raft_consensus.cc:2229] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.691004 27844 raft_consensus.cc:2258] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.690784 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db95b9847807414aa63e686837a0c9c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.691033 27585 raft_consensus.cc:2454] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.691077 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960d68053f9642ff82c9c30983973e93" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.691125 27450 raft_consensus.cc:2454] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.691192 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960d68053f9642ff82c9c30983973e93" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.691226 27449 raft_consensus.cc:3037] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.691284 27844 ts_tablet_manager.cc:1875] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.691408 27587 raft_consensus.cc:2454] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.690804 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 4210a30264df45c89eec0a6bebe3b46c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.691788 27449 raft_consensus.cc:2454] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.692095 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.692155 27453 raft_consensus.cc:2454] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.692334 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db95b9847807414aa63e686837a0c9c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.692418 27586 raft_consensus.cc:3037] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.692441 27844 ts_tablet_manager.cc:1888] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.692481 27844 log.cc:1196] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/faf4927166d74721a6d1890adadd3019
I20240503 02:09:07.692579 27884 raft_consensus.cc:504] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.692646 27884 leader_election.cc:288] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.692646 27844 ts_tablet_manager.cc:1909] T faf4927166d74721a6d1890adadd3019 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.692883 27844 tablet_replica.cc:330] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.692930 27844 raft_consensus.cc:2229] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.692958 27844 raft_consensus.cc:2258] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.692970 27586 raft_consensus.cc:2454] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.693106 27524 leader_election.cc:302] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.693123 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet faf4927166d74721a6d1890adadd3019 (table default.Table_2691a07efb4c43e698e53947e6b90445 [id=10fd8d0ff7104ffe99d73720ff9a5f76]) successfully deleted
I20240503 02:09:07.693207 27917 tablet.cc:1722] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:07.693471 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.693532 27453 raft_consensus.cc:3037] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.693540 27848 raft_consensus.cc:2781] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.693588 27848 raft_consensus.cc:686] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.693619 27848 consensus_queue.cc:237] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.693923 27844 ts_tablet_manager.cc:1875] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.694013 27453 raft_consensus.cc:2454] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.694097 27386 leader_election.cc:302] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.694164 28054 raft_consensus.cc:2781] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.694227 28054 raft_consensus.cc:686] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.694272 28054 consensus_queue.cc:237] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.694710 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920704b080df46dcb3fbb58646c34a1d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.694763 27723 raft_consensus.cc:3037] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.695036 27844 ts_tablet_manager.cc:1888] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.695068 27844 log.cc:1196] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/506cf6ad0d2645a491252b525117e9ad
I20240503 02:09:07.695187 27844 ts_tablet_manager.cc:1909] T 506cf6ad0d2645a491252b525117e9ad P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.695225 27723 raft_consensus.cc:2454] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.695264 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "960d68053f9642ff82c9c30983973e93" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.695313 27722 raft_consensus.cc:3037] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.695463 27844 tablet_replica.cc:330] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.695509 27844 raft_consensus.cc:2229] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.695573 27844 raft_consensus.cc:2258] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.695597 27523 leader_election.cc:302] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.695684 28007 raft_consensus.cc:2781] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.695719 28007 raft_consensus.cc:686] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.695752 28007 consensus_queue.cc:237] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.695876 27722 raft_consensus.cc:2454] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.696094 27917 tablet.cc:1722] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.690184 27837 consensus_queue.cc:260] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.697149 27983 tablet.cc:1722] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.697183 27837 ts_tablet_manager.cc:1398] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:07.697237 27837 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.697281 28048 mvcc.cc:205] Tried to move back new op lower bound from 7023419996827705344 to 7023419996098412544. Current Snapshot: MvccSnapshot[applied={T|T < 7023419996827705344}]
I20240503 02:09:07.697583 27837 tablet_bootstrap.cc:654] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.698163 27892 tablet.cc:1722] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.698190 27907 mvcc.cc:205] Tried to move back new op lower bound from 7023419996827705344 to 7023419996098412544. Current Snapshot: MvccSnapshot[applied={T|T < 7023419996827705344}]
I20240503 02:09:07.698544 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 506cf6ad0d2645a491252b525117e9ad (table default.Table_e71b7dabce8240cc94bfdbe2ebff7bcc [id=737d279d213d4e598c098e4a32020f1f]) successfully deleted
I20240503 02:09:07.699359 27839 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.699404 27839 ts_tablet_manager.cc:1367] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.000s
I20240503 02:09:07.699431 27917 tablet.cc:1722] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.699529 27839 raft_consensus.cc:348] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.699587 27839 raft_consensus.cc:374] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.699611 27839 raft_consensus.cc:729] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.699643 27839 consensus_queue.cc:260] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.699718 27839 ts_tablet_manager.cc:1398] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.699680 27125 catalog_manager.cc:5484] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.699767 27839 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.700054 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 (DEFAULT_TABLE table=default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.700083 27839 tablet_bootstrap.cc:654] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.700135 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b3187a99554dedaaeeb66d94603fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.700762 27127 catalog_manager.cc:5484] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.701035 27837 tablet_bootstrap.cc:492] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.701036 27839 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.701071 27837 ts_tablet_manager.cc:1367] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.701076 27839 ts_tablet_manager.cc:1367] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.701190 27837 raft_consensus.cc:348] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.701200 27839 raft_consensus.cc:348] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.701236 27837 raft_consensus.cc:374] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.701252 27839 raft_consensus.cc:374] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.701254 27837 raft_consensus.cc:729] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.701269 27839 raft_consensus.cc:729] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.701292 27837 consensus_queue.cc:260] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.701304 27839 consensus_queue.cc:260] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.701381 27837 ts_tablet_manager.cc:1398] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.701370 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 (DEFAULT_TABLE table=default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.701424 27837 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.701424 27839 ts_tablet_manager.cc:1398] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.701447 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b3187a99554dedaaeeb66d94603fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.701469 27839 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.701707 27837 tablet_bootstrap.cc:654] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.701823 27839 tablet_bootstrap.cc:654] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.702172 27837 tablet_bootstrap.cc:492] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.702206 27837 ts_tablet_manager.cc:1367] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.702311 27837 raft_consensus.cc:348] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.702373 27837 raft_consensus.cc:374] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.702390 27837 raft_consensus.cc:729] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.702431 27837 consensus_queue.cc:260] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.702504 27837 ts_tablet_manager.cc:1398] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.702495 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e8c66d743724b81ab178b9bf0405b90 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.702548 27837 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.702908 27844 ts_tablet_manager.cc:1875] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.702966 27837 tablet_bootstrap.cc:654] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.703431 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 (DEFAULT_TABLE table=default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.703575 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b3187a99554dedaaeeb66d94603fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.703752 27837 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.703785 27837 ts_tablet_manager.cc:1367] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.703898 27837 raft_consensus.cc:348] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.703960 27837 raft_consensus.cc:374] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.703977 27837 raft_consensus.cc:729] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.704020 27837 consensus_queue.cc:260] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.704097 27837 ts_tablet_manager.cc:1398] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.704102 27844 ts_tablet_manager.cc:1888] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.704166 27837 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.704501 27839 tablet_bootstrap.cc:492] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.704538 27839 ts_tablet_manager.cc:1367] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.704618 27837 tablet_bootstrap.cc:654] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.704730 27839 raft_consensus.cc:348] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.704779 27839 raft_consensus.cc:374] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.704790 27839 raft_consensus.cc:729] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.704906 27844 log.cc:1196] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/97b7b16f82564b1ba88df6c72e41eb62
I20240503 02:09:07.704826 27839 consensus_queue.cc:260] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.705202 27839 ts_tablet_manager.cc:1398] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.705257 27844 ts_tablet_manager.cc:1909] T 97b7b16f82564b1ba88df6c72e41eb62 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.705277 27839 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.705528 27844 tablet_replica.cc:330] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.705634 27844 raft_consensus.cc:2229] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.705678 27844 raft_consensus.cc:2258] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.705703 27839 tablet_bootstrap.cc:654] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.705760 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 97b7b16f82564b1ba88df6c72e41eb62 (table default.Table_786b4055ce5140d08d17981aef2d77b2 [id=3fc74e77e2e34caa8d28d5428b5806ea]) successfully deleted
I20240503 02:09:07.705911 27844 ts_tablet_manager.cc:1875] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.706207 27126 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446] alter complete (version 1)
I20240503 02:09:07.706334 27837 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.706380 27837 ts_tablet_manager.cc:1367] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.706409 27839 tablet_bootstrap.cc:492] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.706441 27839 ts_tablet_manager.cc:1367] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.706494 27837 raft_consensus.cc:348] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.706542 27837 raft_consensus.cc:374] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.706559 27837 raft_consensus.cc:729] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.706548 27839 raft_consensus.cc:348] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.706593 27839 raft_consensus.cc:374] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.706611 27839 raft_consensus.cc:729] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.706595 27837 consensus_queue.cc:260] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.706645 27839 consensus_queue.cc:260] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.706666 27837 ts_tablet_manager.cc:1398] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.706715 27839 ts_tablet_manager.cc:1398] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.706719 27837 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.706782 27839 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.707082 27839 tablet_bootstrap.cc:654] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.707079 27837 tablet_bootstrap.cc:654] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.707248 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet d2c180bd912148dfbb6cfc373d1a91ab
I20240503 02:09:07.707441 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 0e8c66d743724b81ab178b9bf0405b90
I20240503 02:09:07.707528 27123 catalog_manager.cc:6554] ReplaceTablet: tablet 0e8c66d743724b81ab178b9bf0405b90 deleted and replaced by tablet 38d89374090345758db303c33b460c9a
I20240503 02:09:07.707578 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.707551 27844 ts_tablet_manager.cc:1888] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.707665 27844 log.cc:1196] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4210a30264df45c89eec0a6bebe3b46c
I20240503 02:09:07.707751 27845 tablet_replica.cc:330] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.707810 27844 ts_tablet_manager.cc:1909] T 4210a30264df45c89eec0a6bebe3b46c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.707840 27845 raft_consensus.cc:2229] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.707871 27845 raft_consensus.cc:2258] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.708120 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 with delete_type TABLET_DATA_DELETED (replaced by tablet 38d89374090345758db303c33b460c9a) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.708160 27839 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.708195 27839 ts_tablet_manager.cc:1367] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.708186 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 with delete_type TABLET_DATA_DELETED (replaced by tablet 38d89374090345758db303c33b460c9a) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.708245 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.708245 27845 ts_tablet_manager.cc:1875] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.708246 27122 master_service.cc:946] ReplaceTablet: received request to replace tablet 51172f389028420fb0690adea9673429 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.708302 27839 raft_consensus.cc:348] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.708348 27839 raft_consensus.cc:374] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.708364 27839 raft_consensus.cc:729] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.708396 27839 consensus_queue.cc:260] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.708458 27839 ts_tablet_manager.cc:1398] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.708498 27839 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.708791 27839 tablet_bootstrap.cc:654] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.709225 27843 tablet_replica.cc:330] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.709281 27843 raft_consensus.cc:2229] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.709311 27843 raft_consensus.cc:2258] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.709448 27839 tablet_bootstrap.cc:492] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.709510 27839 ts_tablet_manager.cc:1367] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.709632 27839 raft_consensus.cc:348] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.709683 27839 raft_consensus.cc:374] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.709700 27839 raft_consensus.cc:729] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.709683 27843 ts_tablet_manager.cc:1875] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.709736 27839 consensus_queue.cc:260] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.709838 27839 ts_tablet_manager.cc:1398] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.709887 27839 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.710059 27127 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530] alter complete (version 1)
I20240503 02:09:07.710188 27839 tablet_bootstrap.cc:654] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.709447 27845 ts_tablet_manager.cc:1888] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.710340 27845 log.cc:1196] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d2c180bd912148dfbb6cfc373d1a91ab
I20240503 02:09:07.710465 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9ff52b758952450dbc6eb3162457000f (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.710496 27845 ts_tablet_manager.cc:1909] T d2c180bd912148dfbb6cfc373d1a91ab P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.710691 27845 tablet_replica.cc:330] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.710734 27845 raft_consensus.cc:2229] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.710762 27845 raft_consensus.cc:2258] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.710827 27843 ts_tablet_manager.cc:1888] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.711071 27843 log.cc:1196] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0e8c66d743724b81ab178b9bf0405b90
I20240503 02:09:07.711136 27837 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.711170 27837 ts_tablet_manager.cc:1367] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.711221 27843 ts_tablet_manager.cc:1909] T 0e8c66d743724b81ab178b9bf0405b90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.711278 27837 raft_consensus.cc:348] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.711328 27837 raft_consensus.cc:374] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.711344 27837 raft_consensus.cc:729] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.711390 27843 tablet_replica.cc:330] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.711390 27837 consensus_queue.cc:260] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.711429 27843 raft_consensus.cc:2229] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.711454 27843 raft_consensus.cc:2258] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.711465 27837 ts_tablet_manager.cc:1398] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.711513 27837 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.711720 27125 catalog_manager.cc:5484] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.711831 27837 tablet_bootstrap.cc:654] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.711829 27843 ts_tablet_manager.cc:1875] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.711834 27125 catalog_manager.cc:5484] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.711886 27125 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.712066 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d2c180bd912148dfbb6cfc373d1a91ab (table default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]) successfully deleted
I20240503 02:09:07.712138 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 38d89374090345758db303c33b460c9a (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.712173 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0e8c66d743724b81ab178b9bf0405b90 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:07.712215 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d89374090345758db303c33b460c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.712322 27126 master_service.cc:946] ReplaceTablet: received request to replace tablet f7430ba7aada43968a0aac0ac10c5f2d from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.712599 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4210a30264df45c89eec0a6bebe3b46c (table default.Table_d313bd2d1c434bbc8d5d364828596066 [id=64630afa2665476592d50c89f1526567]) successfully deleted
I20240503 02:09:07.712666 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet d2c180bd912148dfbb6cfc373d1a91ab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.712680 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 38d89374090345758db303c33b460c9a (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.712778 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d89374090345758db303c33b460c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.712875 27843 ts_tablet_manager.cc:1888] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.712911 27843 log.cc:1196] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d2c180bd912148dfbb6cfc373d1a91ab
I20240503 02:09:07.713034 27843 ts_tablet_manager.cc:1909] T d2c180bd912148dfbb6cfc373d1a91ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.713239 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 38d89374090345758db303c33b460c9a (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.713316 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38d89374090345758db303c33b460c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.713354 27845 ts_tablet_manager.cc:1875] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.713485 27844 tablet_replica.cc:330] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.713536 27844 raft_consensus.cc:2229] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.713562 27844 raft_consensus.cc:2258] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.714251 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c66d743724b81ab178b9bf0405b90 with delete_type TABLET_DATA_DELETED (replaced by tablet 38d89374090345758db303c33b460c9a) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.714272 27844 ts_tablet_manager.cc:1875] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.714390 27845 ts_tablet_manager.cc:1888] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.714426 27845 log.cc:1196] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0e8c66d743724b81ab178b9bf0405b90
I20240503 02:09:07.714560 27845 ts_tablet_manager.cc:1909] T 0e8c66d743724b81ab178b9bf0405b90 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.714745 27127 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc] alter complete (version 1)
I20240503 02:09:07.714866 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet f7430ba7aada43968a0aac0ac10c5f2d
I20240503 02:09:07.714908 27126 catalog_manager.cc:6554] ReplaceTablet: tablet f7430ba7aada43968a0aac0ac10c5f2d deleted and replaced by tablet 9ee074389e564425809ccc2323f114e9
I20240503 02:09:07.715268 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ee074389e564425809ccc2323f114e9) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.715538 27844 ts_tablet_manager.cc:1888] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.715579 27844 log.cc:1196] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d2c180bd912148dfbb6cfc373d1a91ab
I20240503 02:09:07.715643 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d2c180bd912148dfbb6cfc373d1a91ab (table default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]) successfully deleted
I20240503 02:09:07.715715 27844 ts_tablet_manager.cc:1909] T d2c180bd912148dfbb6cfc373d1a91ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.715888 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ee074389e564425809ccc2323f114e9) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.715981 27843 tablet_replica.cc:330] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.715976 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet f7430ba7aada43968a0aac0ac10c5f2d with delete_type TABLET_DATA_DELETED (replaced by tablet 9ee074389e564425809ccc2323f114e9) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.716025 27843 raft_consensus.cc:2229] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.716051 27843 raft_consensus.cc:2258] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.716202 27845 tablet_replica.cc:330] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.716209 27843 ts_tablet_manager.cc:1875] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.716243 27845 raft_consensus.cc:2229] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.716267 27845 raft_consensus.cc:2258] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.716270 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0e8c66d743724b81ab178b9bf0405b90 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:07.716418 27844 tablet_replica.cc:330] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.716459 27844 raft_consensus.cc:2229] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.716674 27844 raft_consensus.cc:2258] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.716734 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d2c180bd912148dfbb6cfc373d1a91ab (table default.Table_9017d82aa4984e29a21642db1dd0c595 [id=2adf26ee10c945bfa34176ee12d80472]) successfully deleted
I20240503 02:09:07.716828 27844 ts_tablet_manager.cc:1875] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.717237 27845 ts_tablet_manager.cc:1875] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.717612 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 51172f389028420fb0690adea9673429
I20240503 02:09:07.717684 27122 catalog_manager.cc:6554] ReplaceTablet: tablet 51172f389028420fb0690adea9673429 deleted and replaced by tablet 630a06d78f984c27903d401de9e7247a
I20240503 02:09:07.717831 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 51172f389028420fb0690adea9673429 with delete_type TABLET_DATA_DELETED (replaced by tablet 630a06d78f984c27903d401de9e7247a) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.717938 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 51172f389028420fb0690adea9673429 with delete_type TABLET_DATA_DELETED (replaced by tablet 630a06d78f984c27903d401de9e7247a) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.718109 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 51172f389028420fb0690adea9673429 with delete_type TABLET_DATA_DELETED (replaced by tablet 630a06d78f984c27903d401de9e7247a) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.718281 27844 ts_tablet_manager.cc:1888] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.718457 27844 log.cc:1196] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0e8c66d743724b81ab178b9bf0405b90
I20240503 02:09:07.718456 27843 ts_tablet_manager.cc:1888] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.718495 27843 log.cc:1196] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f7430ba7aada43968a0aac0ac10c5f2d
I20240503 02:09:07.718619 27844 ts_tablet_manager.cc:1909] T 0e8c66d743724b81ab178b9bf0405b90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.718619 27843 ts_tablet_manager.cc:1909] T f7430ba7aada43968a0aac0ac10c5f2d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.718609 27122 master_service.cc:946] ReplaceTablet: received request to replace tablet af46440213c24b93b3c66fd2e5cd406f from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.718802 27845 ts_tablet_manager.cc:1888] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.718847 27845 log.cc:1196] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f7430ba7aada43968a0aac0ac10c5f2d
I20240503 02:09:07.718847 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0e8c66d743724b81ab178b9bf0405b90 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:07.718950 27843 tablet_replica.cc:330] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.718988 27845 ts_tablet_manager.cc:1909] T f7430ba7aada43968a0aac0ac10c5f2d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.719023 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.718991 27843 raft_consensus.cc:2229] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.719130 27843 raft_consensus.cc:2258] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.719147 27845 tablet_replica.cc:330] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.719180 27845 raft_consensus.cc:2229] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.719203 27845 raft_consensus.cc:2258] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.719198 27844 tablet_replica.cc:330] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.719206 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.719237 27844 raft_consensus.cc:2229] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.719262 27844 raft_consensus.cc:2258] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.719439 27844 ts_tablet_manager.cc:1875] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.719501 27837 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.719540 27837 ts_tablet_manager.cc:1367] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:07.719655 27837 raft_consensus.cc:348] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.719725 27837 raft_consensus.cc:374] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.719744 27837 raft_consensus.cc:729] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.719776 27837 consensus_queue.cc:260] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.719857 27837 ts_tablet_manager.cc:1398] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.719851 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee074389e564425809ccc2323f114e9 (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.719923 27837 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.719934 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee074389e564425809ccc2323f114e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.720211 27837 tablet_bootstrap.cc:654] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.720335 27843 ts_tablet_manager.cc:1875] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.720536 27844 ts_tablet_manager.cc:1888] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.720575 27844 log.cc:1196] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f7430ba7aada43968a0aac0ac10c5f2d
I20240503 02:09:07.720708 27844 ts_tablet_manager.cc:1909] T f7430ba7aada43968a0aac0ac10c5f2d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.720889 27845 ts_tablet_manager.cc:1875] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.720995 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee074389e564425809ccc2323f114e9 (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.721078 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee074389e564425809ccc2323f114e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.721172 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet af46440213c24b93b3c66fd2e5cd406f
I20240503 02:09:07.721190 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 9ee074389e564425809ccc2323f114e9 (DEFAULT_TABLE table=default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.721235 27122 catalog_manager.cc:6554] ReplaceTablet: tablet af46440213c24b93b3c66fd2e5cd406f deleted and replaced by tablet 39ac26cca1be4b7e8cbfa227ab39e38f
I20240503 02:09:07.721266 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ee074389e564425809ccc2323f114e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.721706 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet af46440213c24b93b3c66fd2e5cd406f with delete_type TABLET_DATA_DELETED (replaced by tablet 39ac26cca1be4b7e8cbfa227ab39e38f) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.721987 27845 ts_tablet_manager.cc:1888] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.722030 27845 log.cc:1196] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/51172f389028420fb0690adea9673429
I20240503 02:09:07.722164 27845 ts_tablet_manager.cc:1909] T 51172f389028420fb0690adea9673429 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.722345 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet af46440213c24b93b3c66fd2e5cd406f with delete_type TABLET_DATA_DELETED (replaced by tablet 39ac26cca1be4b7e8cbfa227ab39e38f) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.722496 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 51172f389028420fb0690adea9673429 (table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]) successfully deleted
I20240503 02:09:07.722574 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet af46440213c24b93b3c66fd2e5cd406f with delete_type TABLET_DATA_DELETED (replaced by tablet 39ac26cca1be4b7e8cbfa227ab39e38f) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.722625 27845 tablet_replica.cc:330] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.722692 27845 raft_consensus.cc:2229] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.722726 27845 raft_consensus.cc:2258] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.722764 27844 tablet_replica.cc:330] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.722801 27844 raft_consensus.cc:2229] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.722888 27844 raft_consensus.cc:2258] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.722961 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f7430ba7aada43968a0aac0ac10c5f2d (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.723039 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 630a06d78f984c27903d401de9e7247a (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.723116 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630a06d78f984c27903d401de9e7247a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.723109 27845 ts_tablet_manager.cc:1875] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.723234 27126 master_service.cc:946] ReplaceTablet: received request to replace tablet f7430ba7aada43968a0aac0ac10c5f2d from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.723428 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 630a06d78f984c27903d401de9e7247a (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.723517 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630a06d78f984c27903d401de9e7247a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.724206 27845 ts_tablet_manager.cc:1888] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.724246 27845 log.cc:1196] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/af46440213c24b93b3c66fd2e5cd406f
I20240503 02:09:07.724392 27844 ts_tablet_manager.cc:1875] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.724423 27845 ts_tablet_manager.cc:1909] T af46440213c24b93b3c66fd2e5cd406f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.724596 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 630a06d78f984c27903d401de9e7247a (DEFAULT_TABLE table=default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.724685 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 630a06d78f984c27903d401de9e7247a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.724700 27837 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.724735 27837 ts_tablet_manager.cc:1367] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:07.724751 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.724819 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ac26cca1be4b7e8cbfa227ab39e38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.724843 27837 raft_consensus.cc:348] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.724901 27837 raft_consensus.cc:374] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.724918 27837 raft_consensus.cc:729] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.724952 27837 consensus_queue.cc:260] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.725020 27837 ts_tablet_manager.cc:1398] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.725030 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.725095 27837 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.725111 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ac26cca1be4b7e8cbfa227ab39e38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.725554 27837 tablet_bootstrap.cc:654] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.725841 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.725922 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39ac26cca1be4b7e8cbfa227ab39e38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.726110 27837 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.726146 27837 ts_tablet_manager.cc:1367] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.726263 27837 raft_consensus.cc:348] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.726311 27837 raft_consensus.cc:374] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.726328 27837 raft_consensus.cc:729] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.726352 27837 consensus_queue.cc:260] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.726420 27837 ts_tablet_manager.cc:1398] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.726444 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.726466 27837 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.726800 27837 tablet_bootstrap.cc:654] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.726917 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
W20240503 02:09:07.726984 27126 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 9ee074389e564425809ccc2323f114e9 consensus state has no leader
I20240503 02:09:07.727461 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.727485 27837 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.727545 27837 ts_tablet_manager.cc:1367] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.727653 27837 raft_consensus.cc:348] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.727710 27837 raft_consensus.cc:374] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.727712 27843 ts_tablet_manager.cc:1888] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.727735 27837 raft_consensus.cc:729] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.727754 27843 log.cc:1196] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/51172f389028420fb0690adea9673429
I20240503 02:09:07.727782 27837 consensus_queue.cc:260] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.727880 27837 ts_tablet_manager.cc:1398] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.727892 27843 ts_tablet_manager.cc:1909] T 51172f389028420fb0690adea9673429 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.727941 27837 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.728188 27839 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.728228 27839 ts_tablet_manager.cc:1367] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240503 02:09:07.728350 27837 tablet_bootstrap.cc:654] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.728340 27839 raft_consensus.cc:348] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.728392 27839 raft_consensus.cc:374] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.728395 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 51172f389028420fb0690adea9673429 (table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]) successfully deleted
I20240503 02:09:07.728413 27839 raft_consensus.cc:729] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.728458 27843 tablet_replica.cc:330] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.728448 27839 consensus_queue.cc:260] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.728498 27843 raft_consensus.cc:2229] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.728523 27844 ts_tablet_manager.cc:1888] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.728533 27839 ts_tablet_manager.cc:1398] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.728554 27843 raft_consensus.cc:2258] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.728564 27844 log.cc:1196] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/51172f389028420fb0690adea9673429
I20240503 02:09:07.728590 27839 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.728900 27839 tablet_bootstrap.cc:654] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.729140 27843 ts_tablet_manager.cc:1875] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.729286 27837 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.729406 27837 ts_tablet_manager.cc:1367] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.729478 27844 ts_tablet_manager.cc:1909] T 51172f389028420fb0690adea9673429 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.729547 27837 raft_consensus.cc:348] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.729593 27837 raft_consensus.cc:374] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.729610 27837 raft_consensus.cc:729] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.729648 27837 consensus_queue.cc:260] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.729714 27844 tablet_replica.cc:330] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.729758 27844 raft_consensus.cc:2229] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.729796 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 51172f389028420fb0690adea9673429 (table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]) successfully deleted
I20240503 02:09:07.729816 27844 raft_consensus.cc:2258] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.729722 27837 ts_tablet_manager.cc:1398] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.730262 27839 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.730305 27839 ts_tablet_manager.cc:1367] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.730394 27844 ts_tablet_manager.cc:1875] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.730427 27839 raft_consensus.cc:348] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.730489 27839 raft_consensus.cc:374] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.730506 27839 raft_consensus.cc:729] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.730541 27839 consensus_queue.cc:260] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.730615 27839 ts_tablet_manager.cc:1398] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.730672 27839 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.730966 27839 tablet_bootstrap.cc:654] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.731218 27843 ts_tablet_manager.cc:1888] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.731258 27843 log.cc:1196] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/af46440213c24b93b3c66fd2e5cd406f
I20240503 02:09:07.731408 27844 ts_tablet_manager.cc:1888] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.731431 27839 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.731449 27844 log.cc:1196] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/af46440213c24b93b3c66fd2e5cd406f
I20240503 02:09:07.731463 27839 ts_tablet_manager.cc:1367] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.731559 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
W20240503 02:09:07.731628 27126 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 39ac26cca1be4b7e8cbfa227ab39e38f consensus state has no leader
I20240503 02:09:07.731634 27844 ts_tablet_manager.cc:1909] T af46440213c24b93b3c66fd2e5cd406f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.731652 27839 raft_consensus.cc:348] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.731710 27839 raft_consensus.cc:374] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.731729 27839 raft_consensus.cc:729] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.731722 27843 ts_tablet_manager.cc:1909] T af46440213c24b93b3c66fd2e5cd406f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.731788 27839 consensus_queue.cc:260] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.731884 27839 ts_tablet_manager.cc:1398] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.731937 27839 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.731966 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.732110 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet af46440213c24b93b3c66fd2e5cd406f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:07.732231 27839 tablet_bootstrap.cc:654] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.732618 27838 tablet_bootstrap.cc:492] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.732671 27838 ts_tablet_manager.cc:1367] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.001s
I20240503 02:09:07.732791 27838 raft_consensus.cc:348] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.732846 27838 raft_consensus.cc:374] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.732864 27838 raft_consensus.cc:729] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.732895 27838 consensus_queue.cc:260] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.732950 27839 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.732992 27839 ts_tablet_manager.cc:1367] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.732991 27838 ts_tablet_manager.cc:1398] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.733114 27839 raft_consensus.cc:348] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.733135 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.733161 27839 raft_consensus.cc:374] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.733181 27839 raft_consensus.cc:729] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.733212 27839 consensus_queue.cc:260] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.733258 27912 tablet.cc:1722] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.733281 27839 ts_tablet_manager.cc:1398] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.733493 27838 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.733793 27839 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.733822 27838 tablet_bootstrap.cc:654] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.734129 27839 tablet_bootstrap.cc:654] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.734584 27838 tablet_bootstrap.cc:492] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.734627 27838 ts_tablet_manager.cc:1367] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.734755 27838 raft_consensus.cc:348] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.734819 27838 raft_consensus.cc:374] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.734838 27838 raft_consensus.cc:729] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.734867 27838 consensus_queue.cc:260] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.734948 27838 ts_tablet_manager.cc:1398] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.734988 27838 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.735291 27838 tablet_bootstrap.cc:654] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.735673 27839 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.735715 27839 ts_tablet_manager.cc:1367] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.735838 27839 raft_consensus.cc:348] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.735886 27839 raft_consensus.cc:374] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.735900 27839 raft_consensus.cc:729] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.735934 27839 consensus_queue.cc:260] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.736032 27839 ts_tablet_manager.cc:1398] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.736228 27838 tablet_bootstrap.cc:492] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.736274 27838 ts_tablet_manager.cc:1367] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.736397 27838 raft_consensus.cc:348] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.736456 27838 raft_consensus.cc:374] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.736474 27838 raft_consensus.cc:729] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.736510 27838 consensus_queue.cc:260] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.736588 27838 ts_tablet_manager.cc:1398] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.736656 27838 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.736959 27838 tablet_bootstrap.cc:654] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.737596 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.737902 27838 tablet_bootstrap.cc:492] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.737947 27838 ts_tablet_manager.cc:1367] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.738056 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.738080 27838 raft_consensus.cc:348] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.738134 27838 raft_consensus.cc:374] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.738153 27838 raft_consensus.cc:729] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.738188 27838 consensus_queue.cc:260] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.738279 27838 ts_tablet_manager.cc:1398] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.738330 27838 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.738533 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.738921 27838 tablet_bootstrap.cc:654] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.738968 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]). Expected version 1 got 0
I20240503 02:09:07.739346 27838 tablet_bootstrap.cc:492] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.739383 27838 ts_tablet_manager.cc:1367] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.739487 27838 raft_consensus.cc:348] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.739529 27838 raft_consensus.cc:374] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.739545 27838 raft_consensus.cc:729] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.739594 27838 consensus_queue.cc:260] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.739665 27838 ts_tablet_manager.cc:1398] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.739737 27838 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.740106 27838 tablet_bootstrap.cc:654] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.740712 27838 tablet_bootstrap.cc:492] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.740749 27838 ts_tablet_manager.cc:1367] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.740869 27838 raft_consensus.cc:348] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.740916 27838 raft_consensus.cc:374] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.740929 27838 raft_consensus.cc:729] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.740963 27838 consensus_queue.cc:260] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.741055 27838 ts_tablet_manager.cc:1398] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.741302 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.741817 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.788741 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--webserver_interface=127.26.117.60
--webserver_port=44717
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20240503 02:09:07.807235 27451 raft_consensus.cc:1261] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.807245 27722 raft_consensus.cc:1261] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.807451 27884 consensus_queue.cc:1035] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.807564 28007 consensus_queue.cc:1035] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.808722 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.808856 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]). Expected version 1 got 0
I20240503 02:09:07.809234 27917 tablet.cc:1722] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.810882 27893 tablet.cc:1722] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.811417 28054 raft_consensus.cc:482] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.811473 28054 raft_consensus.cc:504] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.811576 28054 leader_election.cc:288] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.811729 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ffcd42bd284ef3ad4d3c49b010bdbd" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.811794 27588 raft_consensus.cc:2454] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.811919 27386 leader_election.cc:302] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.812062 28054 raft_consensus.cc:2781] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.812099 28054 raft_consensus.cc:482] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.812122 28054 raft_consensus.cc:3037] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.812769 28054 raft_consensus.cc:504] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.812884 28054 leader_election.cc:288] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.813042 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ffcd42bd284ef3ad4d3c49b010bdbd" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.813089 27588 raft_consensus.cc:3037] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.813675 27588 raft_consensus.cc:2454] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.813675 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ffcd42bd284ef3ad4d3c49b010bdbd" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.813736 27723 raft_consensus.cc:3037] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.813825 27386 leader_election.cc:302] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.813886 28054 raft_consensus.cc:2781] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.813921 28054 raft_consensus.cc:686] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.813953 28054 consensus_queue.cc:237] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.814133 27912 tablet.cc:1722] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.814314 27723 raft_consensus.cc:2454] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.814389 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ffcd42bd284ef3ad4d3c49b010bdbd" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.814446 27722 raft_consensus.cc:2362] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1. Re-sending same reply.
I20240503 02:09:07.814970 27126 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d] alter complete (version 1)
I20240503 02:09:07.815131 27123 catalog_manager.cc:5484] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.840200 28007 raft_consensus.cc:482] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.840246 28007 raft_consensus.cc:504] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.840332 28007 leader_election.cc:288] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.840454 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ac26cca1be4b7e8cbfa227ab39e38f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.840518 27722 raft_consensus.cc:2454] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.840525 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ac26cca1be4b7e8cbfa227ab39e38f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.840587 27451 raft_consensus.cc:2454] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.840651 27523 leader_election.cc:302] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.840734 28007 raft_consensus.cc:2781] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.840761 28007 raft_consensus.cc:482] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.840715 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_899d032caab54c299113e3e7bf9a64f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.840776 28007 raft_consensus.cc:3037] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:07.840817 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_899d032caab54c299113e3e7bf9a64f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.841346 28007 raft_consensus.cc:504] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.841459 28007 leader_election.cc:288] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.841625 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ac26cca1be4b7e8cbfa227ab39e38f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.841650 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39ac26cca1be4b7e8cbfa227ab39e38f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.841681 27722 raft_consensus.cc:3037] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.841692 27451 raft_consensus.cc:3037] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.842237 27451 raft_consensus.cc:2454] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.842253 27722 raft_consensus.cc:2454] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.842430 27524 leader_election.cc:302] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.842449 28054 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.842494 28054 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.842581 28054 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.842739 28007 raft_consensus.cc:2781] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.842782 28007 raft_consensus.cc:686] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.842818 28007 consensus_queue.cc:237] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.842851 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d89374090345758db303c33b460c9a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.842897 27722 raft_consensus.cc:2454] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.842916 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d89374090345758db303c33b460c9a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.842978 27588 raft_consensus.cc:2454] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.843147 27386 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.843212 28054 raft_consensus.cc:2781] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.843235 28054 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.843254 28054 raft_consensus.cc:3037] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.843807 28054 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.843897 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 85e1f031e13f47c692de9a8a5e256613 (DEFAULT_TABLE table=default.Table_899d032caab54c299113e3e7bf9a64f6 [id=453d4c39809a42f38267db3c4d45748a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.843995 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e1f031e13f47c692de9a8a5e256613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.843986 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 85e1f031e13f47c692de9a8a5e256613 (DEFAULT_TABLE table=default.Table_899d032caab54c299113e3e7bf9a64f6 [id=453d4c39809a42f38267db3c4d45748a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.844062 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e1f031e13f47c692de9a8a5e256613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.844249 27126 catalog_manager.cc:5484] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.844345 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.844560 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d89374090345758db303c33b460c9a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.844612 27588 raft_consensus.cc:3037] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.844990 27839 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.845021 27838 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.845027 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 85e1f031e13f47c692de9a8a5e256613 (DEFAULT_TABLE table=default.Table_899d032caab54c299113e3e7bf9a64f6 [id=453d4c39809a42f38267db3c4d45748a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.845108 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e1f031e13f47c692de9a8a5e256613. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.845134 27588 raft_consensus.cc:2454] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.845350 27839 tablet_bootstrap.cc:654] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.845383 27838 tablet_bootstrap.cc:654] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.845758 27386 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.845778 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_fa7a4681754d43d7957e168595ad8bc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.845871 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa7a4681754d43d7957e168595ad8bc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.845990 27852 raft_consensus.cc:2781] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.846028 27852 raft_consensus.cc:686] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.846062 27852 consensus_queue.cc:237] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.846302 27837 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.843905 28054 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.846441 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38d89374090345758db303c33b460c9a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.846493 27722 raft_consensus.cc:3037] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.846554 27837 tablet_bootstrap.cc:654] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.846809 27838 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.846846 27838 ts_tablet_manager.cc:1367] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.846994 27722 raft_consensus.cc:2454] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.846967 27838 raft_consensus.cc:348] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.847026 27838 raft_consensus.cc:374] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.847046 27838 raft_consensus.cc:729] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.847015 27127 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_c839b3b37e0f40c487bdc964e6a6a999"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:07.847081 27838 consensus_queue.cc:260] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:07.847106 27127 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c839b3b37e0f40c487bdc964e6a6a999 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.847162 27838 ts_tablet_manager.cc:1398] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.847225 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.847290 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.847227 27125 catalog_manager.cc:5484] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.847741 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_4d99d803f2ac4b599aea7043cceb1506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.847828 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d99d803f2ac4b599aea7043cceb1506 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.848193 27837 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.848239 27837 ts_tablet_manager.cc:1367] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.848320 27837 raft_consensus.cc:348] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.848374 27837 raft_consensus.cc:374] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.848392 27837 raft_consensus.cc:729] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.848426 27837 consensus_queue.cc:260] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.848515 27837 ts_tablet_manager.cc:1398] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.849261 27839 tablet_bootstrap.cc:492] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.849308 27839 ts_tablet_manager.cc:1367] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.849464 27839 raft_consensus.cc:348] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.849529 27839 raft_consensus.cc:374] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.849545 27839 raft_consensus.cc:729] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.849591 27839 consensus_queue.cc:260] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.849674 27839 ts_tablet_manager.cc:1398] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.851491 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_440ae0e7eec44ba28808051dfb31ee04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.851588 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_440ae0e7eec44ba28808051dfb31ee04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.852424 27563 tablet_service.cc:1467] Processing CreateTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d (DEFAULT_TABLE table=default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e [id=0e67e01150414cfda8f556738345b4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.852473 27425 tablet_service.cc:1467] Processing CreateTablet for tablet e14a135e07464f7aaaf9179806eb5224 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.852526 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccd6dc9cfb834653ae08c29ff0f4848d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.852507 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 89440f04f9d349a187c296efab0816d2 (DEFAULT_TABLE table=default.Table_fa7a4681754d43d7957e168595ad8bc7 [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.852547 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14a135e07464f7aaaf9179806eb5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.852597 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89440f04f9d349a187c296efab0816d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.853399 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d (DEFAULT_TABLE table=default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e [id=0e67e01150414cfda8f556738345b4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.853489 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccd6dc9cfb834653ae08c29ff0f4848d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.853574 27564 tablet_service.cc:1467] Processing CreateTablet for tablet e14a135e07464f7aaaf9179806eb5224 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.853646 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14a135e07464f7aaaf9179806eb5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.854039 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_b5b9dfae61664236992fba8ef989ac69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.854133 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5b9dfae61664236992fba8ef989ac69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.854254 27127 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_d1dccb2cfe654cce8410d7503dbf8985"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.854318 27127 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1dccb2cfe654cce8410d7503dbf8985 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.854525 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.854604 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dc7b64b7d4484296de6b8fc78e2705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.854631 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 89440f04f9d349a187c296efab0816d2 (DEFAULT_TABLE table=default.Table_fa7a4681754d43d7957e168595ad8bc7 [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.854722 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89440f04f9d349a187c296efab0816d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.855573 27837 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.855782 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.855855 27837 tablet_bootstrap.cc:654] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.855881 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dc7b64b7d4484296de6b8fc78e2705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.856173 27699 tablet_service.cc:1467] Processing CreateTablet for tablet e14a135e07464f7aaaf9179806eb5224 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.856194 27122 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f2ad1746d1bc42c3991e88f39e97dfae" } new_table_name: "default.Table_f66ef079a68b476d98d2b2cad2cf6773" modify_external_catalogs: true
I20240503 02:09:07.856252 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e14a135e07464f7aaaf9179806eb5224. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.856319 27837 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.856350 27837 ts_tablet_manager.cc:1367] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.856454 27837 raft_consensus.cc:348] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.856498 27837 raft_consensus.cc:374] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.856472 27128 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_fa7a4681754d43d7957e168595ad8bc7" } new_table_name: "default.Table_390a24b35e2f40f982412479ea7ebaec" modify_external_catalogs: true
I20240503 02:09:07.856515 27837 raft_consensus.cc:729] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.856550 27837 consensus_queue.cc:260] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.856621 27837 ts_tablet_manager.cc:1398] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.856657 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_51ec2543db0347e398706eccaf98064a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.856751 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51ec2543db0347e398706eccaf98064a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:07.857224 27703 tablet_service.cc:1467] Processing CreateTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d (DEFAULT_TABLE table=default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e [id=0e67e01150414cfda8f556738345b4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.857306 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccd6dc9cfb834653ae08c29ff0f4848d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.858327 27838 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.858660 27838 tablet_bootstrap.cc:654] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.859063 27126 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_86dfc92ff9444e1aab6ee1670c47639a" } new_table_name: "default.Table_bf60bed921ad4219bd0b452cf484c521" modify_external_catalogs: true
I20240503 02:09:07.859160 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 89440f04f9d349a187c296efab0816d2 (DEFAULT_TABLE table=default.Table_fa7a4681754d43d7957e168595ad8bc7 [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.859236 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89440f04f9d349a187c296efab0816d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.859457 27839 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:07.859613 27128 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 89440f04f9d349a187c296efab0816d2 consensus state has no leader
I20240503 02:09:07.859802 27839 tablet_bootstrap.cc:654] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.860173 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.860246 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42dc7b64b7d4484296de6b8fc78e2705. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.860801 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 16577f3067c04965887c8b3db12795e4 (DEFAULT_TABLE table=default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.860869 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16577f3067c04965887c8b3db12795e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.861796 27837 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.861858 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 16577f3067c04965887c8b3db12795e4 (DEFAULT_TABLE table=default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.861935 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16577f3067c04965887c8b3db12795e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.862124 27837 tablet_bootstrap.cc:654] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.862452 27722 raft_consensus.cc:1261] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.862751 28007 consensus_queue.cc:1035] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.862857 27451 raft_consensus.cc:1261] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.863031 27699 tablet_service.cc:1467] Processing CreateTablet for tablet da16da72302e49568ea8d2967f0d2e7a (DEFAULT_TABLE table=default.Table_51ec2543db0347e398706eccaf98064a [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.863075 27700 tablet_service.cc:1467] Processing CreateTablet for tablet ec323adbf97f48428c1e062aeadb9b88 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.863130 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da16da72302e49568ea8d2967f0d2e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.863039 27701 tablet_service.cc:1467] Processing CreateTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 (DEFAULT_TABLE table=default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.863234 27566 tablet_service.cc:1467] Processing CreateTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 (DEFAULT_TABLE table=default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.863253 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5daed7b8f00446b8d94d71b1705ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.863303 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5daed7b8f00446b8d94d71b1705ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.863341 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_51ec2543db0347e398706eccaf98064a" } new_table_name: "default.Table_dd793faec53545858dfe59dce7a5066c" modify_external_catalogs: true
I20240503 02:09:07.864102 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 16577f3067c04965887c8b3db12795e4 (DEFAULT_TABLE table=default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.864181 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ec323adbf97f48428c1e062aeadb9b88 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.864178 27126 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_899d032caab54c299113e3e7bf9a64f6" } new_table_name: "default.Table_c456a5e68e2f4baaaec502f69453d04b" modify_external_catalogs: true
I20240503 02:09:07.864217 27564 tablet_service.cc:1467] Processing CreateTablet for tablet ec323adbf97f48428c1e062aeadb9b88 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.864257 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec323adbf97f48428c1e062aeadb9b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.864277 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec323adbf97f48428c1e062aeadb9b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.864998 27451 raft_consensus.cc:1261] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.865185 27904 consensus_queue.cc:1035] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.865248 27425 tablet_service.cc:1467] Processing CreateTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 (DEFAULT_TABLE table=default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.865278 27839 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.865296 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec323adbf97f48428c1e062aeadb9b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.865307 27839 ts_tablet_manager.cc:1367] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:07.865454 27839 raft_consensus.cc:348] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.865516 27839 raft_consensus.cc:374] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.865535 27839 raft_consensus.cc:729] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.865561 27839 consensus_queue.cc:260] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.865638 27839 ts_tablet_manager.cc:1398] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.865680 27839 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.865736 27429 tablet_service.cc:1467] Processing CreateTablet for tablet da16da72302e49568ea8d2967f0d2e7a (DEFAULT_TABLE table=default.Table_51ec2543db0347e398706eccaf98064a [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.865792 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5daed7b8f00446b8d94d71b1705ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.865839 27563 tablet_service.cc:1467] Processing CreateTablet for tablet da16da72302e49568ea8d2967f0d2e7a (DEFAULT_TABLE table=default.Table_51ec2543db0347e398706eccaf98064a [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.865918 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da16da72302e49568ea8d2967f0d2e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.866747 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da16da72302e49568ea8d2967f0d2e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.866897 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16577f3067c04965887c8b3db12795e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.867082 28007 consensus_queue.cc:1035] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.867995 27987 mvcc.cc:205] Tried to move back new op lower bound from 7023419997639688192 to 7023419996961988608. Current Snapshot: MvccSnapshot[applied={T|T < 7023419997639688192}]
I20240503 02:09:07.868038 27839 tablet_bootstrap.cc:654] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.868464 27848 raft_consensus.cc:482] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.868510 27848 raft_consensus.cc:504] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.868597 27848 leader_election.cc:288] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.868783 27588 raft_consensus.cc:1261] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.868834 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e419af3f80e40b8a1acdfb71550ee91" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.868896 27453 raft_consensus.cc:2454] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.869011 27853 consensus_queue.cc:1035] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.869149 27917 tablet.cc:1722] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.869210 27524 leader_election.cc:302] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.869283 27884 raft_consensus.cc:2781] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.869307 27884 raft_consensus.cc:482] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.869338 27884 raft_consensus.cc:3037] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.869534 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 920704b080df46dcb3fbb58646c34a1d (table default.Table_f66ef079a68b476d98d2b2cad2cf6773 [id=33435142c7594d9ea99fbf83135cae2c]). Expected version 1 got 0
I20240503 02:09:07.869580 27912 tablet.cc:1722] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20240503 02:09:07.869516 27125 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet da16da72302e49568ea8d2967f0d2e7a consensus state has no leader
I20240503 02:09:07.869882 27125 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f44cea6f7c3d4047bbabe392bba65b57" } modify_external_catalogs: true
I20240503 02:09:07.869899 27884 raft_consensus.cc:504] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.869933 27125 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f44cea6f7c3d4047bbabe392bba65b57" } modify_external_catalogs: true
I20240503 02:09:07.869992 27884 leader_election.cc:288] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.870002 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 920704b080df46dcb3fbb58646c34a1d (table default.Table_f66ef079a68b476d98d2b2cad2cf6773 [id=33435142c7594d9ea99fbf83135cae2c]). Expected version 1 got 0
I20240503 02:09:07.870271 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e419af3f80e40b8a1acdfb71550ee91" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.870324 27453 raft_consensus.cc:3037] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:07.869552 27126 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85e1f031e13f47c692de9a8a5e256613 consensus state has no leader
I20240503 02:09:07.870553 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e419af3f80e40b8a1acdfb71550ee91" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.870607 27723 raft_consensus.cc:2454] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.870683 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e419af3f80e40b8a1acdfb71550ee91" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.870718 27721 raft_consensus.cc:3037] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.870847 27893 tablet.cc:1722] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.870909 27453 raft_consensus.cc:2454] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.870952 27891 tablet.cc:1722] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.871212 27524 leader_election.cc:302] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.871269 27848 raft_consensus.cc:2781] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.871309 27848 raft_consensus.cc:686] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.871309 27721 raft_consensus.cc:2454] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.871343 27848 consensus_queue.cc:237] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.869938 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:07.872058 27126 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f66ef079a68b476d98d2b2cad2cf6773" } modify_external_catalogs: true
I20240503 02:09:07.872154 27126 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_f66ef079a68b476d98d2b2cad2cf6773" } modify_external_catalogs: true
I20240503 02:09:07.872495 27892 tablet.cc:1722] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.875485 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b5b9dfae61664236992fba8ef989ac69" } modify_external_catalogs: true
I20240503 02:09:07.875547 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b5b9dfae61664236992fba8ef989ac69" } modify_external_catalogs: true
I20240503 02:09:07.875789 27893 tablet.cc:1722] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.877053 27125 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 53b3187a99554dedaaeeb66d94603fe1
I20240503 02:09:07.877367 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.877460 27844 tablet_replica.cc:330] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.877485 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:07.877516 27844 raft_consensus.cc:2229] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.877547 27844 raft_consensus.cc:2258] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.877681 27844 ts_tablet_manager.cc:1875] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.877669 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.877744 27843 tablet_replica.cc:330] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.877799 27843 raft_consensus.cc:2229] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.877825 27843 raft_consensus.cc:2258] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.877985 27843 ts_tablet_manager.cc:1875] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.878329 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet f5daed7b8f00446b8d94d71b1705ac01
I20240503 02:09:07.878600 27128 catalog_manager.cc:5484] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.878799 27844 ts_tablet_manager.cc:1888] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.878841 27844 log.cc:1196] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/53b3187a99554dedaaeeb66d94603fe1
I20240503 02:09:07.878983 27844 ts_tablet_manager.cc:1909] T 53b3187a99554dedaaeeb66d94603fe1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.879082 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.879104 27843 ts_tablet_manager.cc:1888] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.879143 27843 log.cc:1196] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/53b3187a99554dedaaeeb66d94603fe1
I20240503 02:09:07.879160 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.879279 27843 ts_tablet_manager.cc:1909] T 53b3187a99554dedaaeeb66d94603fe1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.879302 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 920704b080df46dcb3fbb58646c34a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.879384 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 920704b080df46dcb3fbb58646c34a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.879412 27844 tablet_replica.cc:330] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.879420 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.879462 27844 raft_consensus.cc:2229] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.879498 27845 tablet_replica.cc:330] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.879509 27844 raft_consensus.cc:2258] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.879549 27845 raft_consensus.cc:2229] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.879546 27126 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_1810f6c6fb234272afd935488b0054cf" } modify_external_catalogs: true
I20240503 02:09:07.879586 27845 raft_consensus.cc:2258] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.879595 27126 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_1810f6c6fb234272afd935488b0054cf" } modify_external_catalogs: true
I20240503 02:09:07.879693 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 53b3187a99554dedaaeeb66d94603fe1 (table default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]) successfully deleted
W20240503 02:09:07.879760 27115 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet f5daed7b8f00446b8d94d71b1705ac01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5daed7b8f00446b8d94d71b1705ac01 already in progress: creating tablet
I20240503 02:09:07.879891 27844 ts_tablet_manager.cc:1875] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.879900 27845 ts_tablet_manager.cc:1875] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.880978 27844 ts_tablet_manager.cc:1888] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.880978 27845 ts_tablet_manager.cc:1888] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.881034 27845 log.cc:1196] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.881034 27844 log.cc:1196] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.881207 27845 ts_tablet_manager.cc:1909] T 920704b080df46dcb3fbb58646c34a1d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.881207 27844 ts_tablet_manager.cc:1909] T 920704b080df46dcb3fbb58646c34a1d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.881583 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 53b3187a99554dedaaeeb66d94603fe1 (table default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]) successfully deleted
I20240503 02:09:07.881709 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.881753 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 920704b080df46dcb3fbb58646c34a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.881808 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 920704b080df46dcb3fbb58646c34a1d (table default.Table_f66ef079a68b476d98d2b2cad2cf6773 [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.881829 27843 tablet_replica.cc:330] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
W20240503 02:09:07.881877 27118 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet f5daed7b8f00446b8d94d71b1705ac01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5daed7b8f00446b8d94d71b1705ac01 already in progress: creating tablet
I20240503 02:09:07.881886 27843 raft_consensus.cc:2229] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.881918 27843 raft_consensus.cc:2258] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.881932 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 920704b080df46dcb3fbb58646c34a1d (table default.Table_f66ef079a68b476d98d2b2cad2cf6773 [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.882077 27843 ts_tablet_manager.cc:1875] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.882985 27843 ts_tablet_manager.cc:1888] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.883028 27843 log.cc:1196] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.883183 27843 ts_tablet_manager.cc:1909] T 920704b080df46dcb3fbb58646c34a1d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.883229 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_3925291fb0534d3582f1823d99624626"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:07.883327 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3925291fb0534d3582f1823d99624626 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:07.883626 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet f5daed7b8f00446b8d94d71b1705ac01 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5daed7b8f00446b8d94d71b1705ac01 already in progress: creating tablet
I20240503 02:09:07.883687 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 920704b080df46dcb3fbb58646c34a1d (table default.Table_f66ef079a68b476d98d2b2cad2cf6773 [id=33435142c7594d9ea99fbf83135cae2c]) successfully deleted
I20240503 02:09:07.883868 27127 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6] alter complete (version 1)
I20240503 02:09:07.884702 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 920704b080df46dcb3fbb58646c34a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.885002 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 9ee074389e564425809ccc2323f114e9
I20240503 02:09:07.885108 27722 raft_consensus.cc:1261] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.885108 27451 raft_consensus.cc:1261] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240503 02:09:07.885272 27118 catalog_manager.cc:4797] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 920704b080df46dcb3fbb58646c34a1d because the tablet was not found. No further retry: Not found: Tablet not found: 920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.885448 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee074389e564425809ccc2323f114e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.885612 27845 tablet_replica.cc:330] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.885658 27845 raft_consensus.cc:2229] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.885689 27845 raft_consensus.cc:2258] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.885259 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 920704b080df46dcb3fbb58646c34a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.885380 27848 consensus_queue.cc:1035] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.885588 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee074389e564425809ccc2323f114e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.885907 27884 consensus_queue.cc:1035] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.885921 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ee074389e564425809ccc2323f114e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.886147 27843 tablet_replica.cc:330] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.886193 27843 raft_consensus.cc:2229] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.886219 27843 raft_consensus.cc:2258] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.886354 27843 ts_tablet_manager.cc:1875] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.886516 27845 ts_tablet_manager.cc:1875] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.886678 27844 tablet_replica.cc:330] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.886734 27844 raft_consensus.cc:2229] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.886761 27844 raft_consensus.cc:2258] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.887017 27844 ts_tablet_manager.cc:1875] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.887384 27843 ts_tablet_manager.cc:1888] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.887426 27843 log.cc:1196] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9ee074389e564425809ccc2323f114e9
I20240503 02:09:07.887581 27843 ts_tablet_manager.cc:1909] T 9ee074389e564425809ccc2323f114e9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
W20240503 02:09:07.887584 27115 catalog_manager.cc:4797] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 920704b080df46dcb3fbb58646c34a1d because the tablet was not found. No further retry: Not found: Tablet not found: 920704b080df46dcb3fbb58646c34a1d
I20240503 02:09:07.887624 27845 ts_tablet_manager.cc:1888] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.887663 27845 log.cc:1196] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9ee074389e564425809ccc2323f114e9
I20240503 02:09:07.887673 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 53b3187a99554dedaaeeb66d94603fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.887821 27845 ts_tablet_manager.cc:1909] T 9ee074389e564425809ccc2323f114e9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.887904 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.888139 27845 tablet_replica.cc:330] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.888191 27845 raft_consensus.cc:2229] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.888221 27845 raft_consensus.cc:2258] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.888418 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.888396 27845 ts_tablet_manager.cc:1875] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.888830 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.888952 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.889006 27917 tablet.cc:1722] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.889250 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:07.889539 27892 tablet.cc:1722] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.889860 27564 tablet_service.cc:1467] Processing CreateTablet for tablet e353654920a44994ae4889144db1b836 (DEFAULT_TABLE table=default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.890302 27701 tablet_service.cc:1467] Processing CreateTablet for tablet e353654920a44994ae4889144db1b836 (DEFAULT_TABLE table=default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.890395 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e353654920a44994ae4889144db1b836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.890643 27845 ts_tablet_manager.cc:1888] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.890698 27845 log.cc:1196] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/53b3187a99554dedaaeeb66d94603fe1
I20240503 02:09:07.890854 27845 ts_tablet_manager.cc:1909] T 53b3187a99554dedaaeeb66d94603fe1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.891433 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e353654920a44994ae4889144db1b836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.891522 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 53b3187a99554dedaaeeb66d94603fe1 (table default.Table_f44cea6f7c3d4047bbabe392bba65b57 [id=2e3f90b6aa27474ab60b2bf20d97f92c]) successfully deleted
I20240503 02:09:07.891788 27429 tablet_service.cc:1467] Processing CreateTablet for tablet e353654920a44994ae4889144db1b836 (DEFAULT_TABLE table=default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.891875 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e353654920a44994ae4889144db1b836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.893190 27844 ts_tablet_manager.cc:1888] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.893371 27844 log.cc:1196] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9ee074389e564425809ccc2323f114e9
I20240503 02:09:07.893610 27844 ts_tablet_manager.cc:1909] T 9ee074389e564425809ccc2323f114e9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.894186 27893 tablet.cc:1722] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.894462 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9ee074389e564425809ccc2323f114e9 (table default.Table_1810f6c6fb234272afd935488b0054cf [id=15ad44d6e11340f28f5fd33dd5b9c91d]) successfully deleted
I20240503 02:09:07.894706 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
W20240503 02:09:07.897197 28089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:07.897405 28089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:07.897469 28089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240503 02:09:07.897658 27838 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.897658 27837 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.897701 27838 ts_tablet_manager.cc:1367] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.001s
I20240503 02:09:07.897701 27837 ts_tablet_manager.cc:1367] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20240503 02:09:07.897791 27838 raft_consensus.cc:348] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.897840 27838 raft_consensus.cc:374] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.897814 27837 raft_consensus.cc:348] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.897857 27838 raft_consensus.cc:729] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.897858 27837 raft_consensus.cc:374] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.897876 27837 raft_consensus.cc:729] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.897891 27838 consensus_queue.cc:260] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.897908 27837 consensus_queue.cc:260] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.897966 27838 ts_tablet_manager.cc:1398] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.897989 27837 ts_tablet_manager.cc:1398] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.898036 27837 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.898042 27838 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.898329 27838 tablet_bootstrap.cc:654] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.898339 27837 tablet_bootstrap.cc:654] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.898919 28007 raft_consensus.cc:482] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.898968 28007 raft_consensus.cc:504] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.899055 28007 leader_election.cc:288] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.899228 27839 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.899258 27839 ts_tablet_manager.cc:1367] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.000s
W20240503 02:09:07.899279 28089 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:07.899335 28089 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:07.899363 28089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:07.899379 28089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:07.899397 28089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20240503 02:09:07.899374 27839 raft_consensus.cc:348] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:07.899430 28089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:07.899454 27839 raft_consensus.cc:374] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.899482 27839 raft_consensus.cc:729] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.899526 27839 consensus_queue.cc:260] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.899611 27839 ts_tablet_manager.cc:1398] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.899677 27839 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.900053 27839 tablet_bootstrap.cc:654] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.900336 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630a06d78f984c27903d401de9e7247a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.900393 27451 raft_consensus.cc:2454] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.900460 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630a06d78f984c27903d401de9e7247a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.900512 27722 raft_consensus.cc:2454] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.900530 27524 leader_election.cc:302] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.900621 28007 raft_consensus.cc:2781] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.900661 28007 raft_consensus.cc:482] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.900678 28007 raft_consensus.cc:3037] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.901185 27839 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.901229 27839 ts_tablet_manager.cc:1367] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.901232 28007 raft_consensus.cc:504] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.901386 28007 leader_election.cc:288] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.901556 28089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.117.60
--webserver_port=44717
--never_fsync=true
--heap_profile_path=/tmp/kudu.28089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:07.901616 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630a06d78f984c27903d401de9e7247a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.901669 27722 raft_consensus.cc:3037] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.901753 28089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:07.902004 28089 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:09:07.902113 27722 raft_consensus.cc:2454] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.902220 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "630a06d78f984c27903d401de9e7247a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.902244 27523 leader_election.cc:302] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.902267 27451 raft_consensus.cc:3037] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.902309 28007 raft_consensus.cc:2781] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.902352 28007 raft_consensus.cc:686] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.902400 28007 consensus_queue.cc:237] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.902798 27451 raft_consensus.cc:2454] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.903214 27839 raft_consensus.cc:348] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.903285 27839 raft_consensus.cc:374] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.903313 27839 raft_consensus.cc:729] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.903353 27839 consensus_queue.cc:260] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.903437 27839 ts_tablet_manager.cc:1398] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.903504 27839 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.903775 27837 tablet_bootstrap.cc:492] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.903813 27839 tablet_bootstrap.cc:654] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.903817 27837 ts_tablet_manager.cc:1367] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:07.904021 27837 raft_consensus.cc:348] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.904071 27837 raft_consensus.cc:374] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.904088 27837 raft_consensus.cc:729] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.904127 27837 consensus_queue.cc:260] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.904215 27837 ts_tablet_manager.cc:1398] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.904902 27837 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.905236 27837 tablet_bootstrap.cc:654] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.905543 27838 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.905586 27838 ts_tablet_manager.cc:1367] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:07.905701 27838 raft_consensus.cc:348] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.905756 27838 raft_consensus.cc:374] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.905778 27838 raft_consensus.cc:729] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.905814 27838 consensus_queue.cc:260] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.905894 27838 ts_tablet_manager.cc:1398] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.905953 27838 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.906229 27838 tablet_bootstrap.cc:654] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.907249 27839 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.907282 27128 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:07.907294 27839 ts_tablet_manager.cc:1367] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.907428 27839 raft_consensus.cc:348] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.907531 27839 raft_consensus.cc:374] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.907552 27839 raft_consensus.cc:729] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.907590 27839 consensus_queue.cc:260] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.907620 27837 tablet_bootstrap.cc:492] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.907671 27839 ts_tablet_manager.cc:1398] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.907692 27837 ts_tablet_manager.cc:1367] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.907725 27839 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.907814 27837 raft_consensus.cc:348] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.907868 27837 raft_consensus.cc:374] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.907886 27837 raft_consensus.cc:729] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.907917 27837 consensus_queue.cc:260] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.908004 27837 ts_tablet_manager.cc:1398] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.907963 27128 catalog_manager.cc:5484] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.908062 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:07.908061 27837 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:07.908254 28121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:07.908351 27837 tablet_bootstrap.cc:654] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.908458 28089 server_base.cc:1034] running on GCE node
I20240503 02:09:07.908566 27839 tablet_bootstrap.cc:654] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:07.908484 28118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:07.908912 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_e4374c6711ad425faf97b03b048df0ec" } modify_external_catalogs: true
I20240503 02:09:07.908963 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_e4374c6711ad425faf97b03b048df0ec" } modify_external_catalogs: true
I20240503 02:09:07.909279 27838 tablet_bootstrap.cc:492] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
W20240503 02:09:07.909775 28119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:07.909777 27838 ts_tablet_manager.cc:1367] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.909925 27838 raft_consensus.cc:348] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.909981 27838 raft_consensus.cc:374] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.909999 27838 raft_consensus.cc:729] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.910032 27838 consensus_queue.cc:260] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.910043 28089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:07.910107 27838 ts_tablet_manager.cc:1398] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.910153 27838 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.910264 28089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:07.910470 27838 tablet_bootstrap.cc:654] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.910766 27127 master_service.cc:946] ReplaceTablet: received request to replace tablet cbeba88b72524e9cb77720cb7ca13597 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.910894 27837 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.910935 27837 ts_tablet_manager.cc:1367] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.911080 27837 raft_consensus.cc:348] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.911137 27837 raft_consensus.cc:374] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.911154 27837 raft_consensus.cc:729] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.911190 27837 consensus_queue.cc:260] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.911262 27837 ts_tablet_manager.cc:1398] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.911310 27837 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.911609 27837 tablet_bootstrap.cc:654] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.911713 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet fe6fb5025f5847f8bb3617c22bdf8c66
I20240503 02:09:07.911981 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.911986 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.912050 27844 tablet_replica.cc:330] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.912096 27844 raft_consensus.cc:2229] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.912092 27839 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.912129 27844 raft_consensus.cc:2258] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.912114 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet fe6fb5025f5847f8bb3617c22bdf8c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.912134 27839 ts_tablet_manager.cc:1367] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:07.912175 27843 tablet_replica.cc:330] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.912217 27843 raft_consensus.cc:2229] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.912247 27843 raft_consensus.cc:2258] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.912248 27839 raft_consensus.cc:348] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.912295 27839 raft_consensus.cc:374] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.912312 27839 raft_consensus.cc:729] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.912096 27845 tablet_replica.cc:330] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.912380 27844 ts_tablet_manager.cc:1875] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.912400 27845 raft_consensus.cc:2229] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.912460 27845 raft_consensus.cc:2258] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.912345 27839 consensus_queue.cc:260] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.912588 27839 ts_tablet_manager.cc:1398] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.912631 27837 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.912644 27839 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.912664 27837 ts_tablet_manager.cc:1367] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:07.912716 27843 ts_tablet_manager.cc:1875] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.912768 27837 raft_consensus.cc:348] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.912817 27837 raft_consensus.cc:374] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.912833 27837 raft_consensus.cc:729] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.912858 27837 consensus_queue.cc:260] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.912927 27837 ts_tablet_manager.cc:1398] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.912968 27839 tablet_bootstrap.cc:654] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.912981 27837 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.913682 28089 hybrid_clock.cc:648] HybridClock initialized: now 1714702147912918 us; error 773 us; skew 500 ppm
I20240503 02:09:07.913868 27843 ts_tablet_manager.cc:1888] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.913883 27844 ts_tablet_manager.cc:1888] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.913910 27843 log.cc:1196] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fe6fb5025f5847f8bb3617c22bdf8c66
I20240503 02:09:07.913926 27844 log.cc:1196] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fe6fb5025f5847f8bb3617c22bdf8c66
I20240503 02:09:07.914063 27837 tablet_bootstrap.cc:654] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.914093 27843 ts_tablet_manager.cc:1909] T fe6fb5025f5847f8bb3617c22bdf8c66 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.914111 27844 ts_tablet_manager.cc:1909] T fe6fb5025f5847f8bb3617c22bdf8c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.914400 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_e4374c6711ad425faf97b03b048df0ec [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.914549 27839 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.914567 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:07.914589 27839 ts_tablet_manager.cc:1367] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:07.914635 27838 tablet_bootstrap.cc:492] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.914670 27838 ts_tablet_manager.cc:1367] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:07.914716 27839 raft_consensus.cc:348] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.914770 27839 raft_consensus.cc:374] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.914754 27838 raft_consensus.cc:348] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.914788 27839 raft_consensus.cc:729] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.914799 27838 raft_consensus.cc:374] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.914821 27838 raft_consensus.cc:729] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.914822 27839 consensus_queue.cc:260] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.914855 27838 consensus_queue.cc:260] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.914897 27839 ts_tablet_manager.cc:1398] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.914925 27838 ts_tablet_manager.cc:1398] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.914950 27839 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.914974 27838 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.915154 27845 ts_tablet_manager.cc:1875] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.915180 27838 tablet_bootstrap.cc:654] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.915246 27839 tablet_bootstrap.cc:654] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.915747 28089 webserver.cc:460] Webserver started at http://127.26.117.60:44717/ using document root <none> and password file <none>
I20240503 02:09:07.915936 28089 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:07.915985 28089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:07.916131 27838 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.916170 27838 ts_tablet_manager.cc:1367] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.916244 27838 raft_consensus.cc:348] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.916290 27838 raft_consensus.cc:374] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.916307 27838 raft_consensus.cc:729] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.916342 27838 consensus_queue.cc:260] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.916404 27837 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.916409 27838 ts_tablet_manager.cc:1398] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.916435 27837 ts_tablet_manager.cc:1367] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.916452 27838 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.916543 27837 raft_consensus.cc:348] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.916595 27837 raft_consensus.cc:374] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.916612 27837 raft_consensus.cc:729] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.916646 27837 consensus_queue.cc:260] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.916721 27838 tablet_bootstrap.cc:654] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.916735 27837 ts_tablet_manager.cc:1398] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.916790 27837 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.916939 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet cbeba88b72524e9cb77720cb7ca13597
I20240503 02:09:07.917011 27127 catalog_manager.cc:6554] ReplaceTablet: tablet cbeba88b72524e9cb77720cb7ca13597 deleted and replaced by tablet b004b105a74e408380a4fe15bacbff03
I20240503 02:09:07.917098 27837 tablet_bootstrap.cc:654] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.917435 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet cbeba88b72524e9cb77720cb7ca13597 with delete_type TABLET_DATA_DELETED (replaced by tablet b004b105a74e408380a4fe15bacbff03) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.917603 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet cbeba88b72524e9cb77720cb7ca13597 with delete_type TABLET_DATA_DELETED (replaced by tablet b004b105a74e408380a4fe15bacbff03) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.917703 27844 tablet_replica.cc:330] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.917716 27839 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.917740 27844 raft_consensus.cc:2229] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.917748 27839 ts_tablet_manager.cc:1367] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.917766 27844 raft_consensus.cc:2258] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.917824 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet cbeba88b72524e9cb77720cb7ca13597 with delete_type TABLET_DATA_DELETED (replaced by tablet b004b105a74e408380a4fe15bacbff03) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.917858 27839 raft_consensus.cc:348] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.917910 27839 raft_consensus.cc:374] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.917927 27839 raft_consensus.cc:729] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.917959 27839 consensus_queue.cc:260] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.918012 27844 ts_tablet_manager.cc:1875] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.918053 27839 ts_tablet_manager.cc:1398] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.918109 27839 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.918418 27839 tablet_bootstrap.cc:654] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.918722 27845 ts_tablet_manager.cc:1888] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:07.918777 27845 log.cc:1196] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fe6fb5025f5847f8bb3617c22bdf8c66
I20240503 02:09:07.918962 27845 ts_tablet_manager.cc:1909] T fe6fb5025f5847f8bb3617c22bdf8c66 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.919063 27844 ts_tablet_manager.cc:1888] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.919111 27844 log.cc:1196] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/cbeba88b72524e9cb77720cb7ca13597
I20240503 02:09:07.919193 27843 tablet_replica.cc:330] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.919200 27845 tablet_replica.cc:330] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.919231 27843 raft_consensus.cc:2229] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.919242 27844 ts_tablet_manager.cc:1909] T cbeba88b72524e9cb77720cb7ca13597 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.919248 27845 raft_consensus.cc:2229] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.919260 27843 raft_consensus.cc:2258] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.919288 27845 pending_rounds.cc:62] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:07.919306 27845 pending_rounds.cc:68] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419996957339648 op_type: NO_OP noop_request { }
I20240503 02:09:07.919338 27845 raft_consensus.cc:2866] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:07.919356 27845 raft_consensus.cc:2258] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.919430 27843 ts_tablet_manager.cc:1875] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.919586 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_e4374c6711ad425faf97b03b048df0ec [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.919714 27845 ts_tablet_manager.cc:1875] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.919826 27838 tablet_bootstrap.cc:492] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.919860 27838 ts_tablet_manager.cc:1367] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:07.919945 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fe6fb5025f5847f8bb3617c22bdf8c66 (table default.Table_e4374c6711ad425faf97b03b048df0ec [id=ffba5f39864e4bc29d61fd32c9f57d31]) successfully deleted
I20240503 02:09:07.919993 27838 raft_consensus.cc:348] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.920043 27838 raft_consensus.cc:374] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.920063 27838 raft_consensus.cc:729] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.920099 27838 consensus_queue.cc:260] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.920181 27838 ts_tablet_manager.cc:1398] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.920238 27838 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.920470 27843 ts_tablet_manager.cc:1888] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.920517 27843 log.cc:1196] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/cbeba88b72524e9cb77720cb7ca13597
I20240503 02:09:07.920552 27838 tablet_bootstrap.cc:654] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.920655 27843 ts_tablet_manager.cc:1909] T cbeba88b72524e9cb77720cb7ca13597 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.921046 27837 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.921064 27128 master_service.cc:946] ReplaceTablet: received request to replace tablet 291c3bbf65b5464cbf8521617fba6365 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.921082 27837 ts_tablet_manager.cc:1367] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:07.921113 27845 ts_tablet_manager.cc:1888] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:07.921151 27845 log.cc:1196] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/cbeba88b72524e9cb77720cb7ca13597
I20240503 02:09:07.921195 27837 raft_consensus.cc:348] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.921254 27837 raft_consensus.cc:374] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.921272 27837 raft_consensus.cc:729] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.921267 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet cbeba88b72524e9cb77720cb7ca13597 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.921343 27845 ts_tablet_manager.cc:1909] T cbeba88b72524e9cb77720cb7ca13597 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.921309 27837 consensus_queue.cc:260] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.921803 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet cbeba88b72524e9cb77720cb7ca13597 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.921835 27837 ts_tablet_manager.cc:1398] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.921895 27837 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.922014 27429 tablet_service.cc:1467] Processing CreateTablet for tablet b004b105a74e408380a4fe15bacbff03 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.922097 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b004b105a74e408380a4fe15bacbff03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.922200 27837 tablet_bootstrap.cc:654] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.922210 27564 tablet_service.cc:1467] Processing CreateTablet for tablet b004b105a74e408380a4fe15bacbff03 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.922300 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b004b105a74e408380a4fe15bacbff03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.922366 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet cbeba88b72524e9cb77720cb7ca13597 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:07.922554 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.922993 28089 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:07.923132 27701 tablet_service.cc:1467] Processing CreateTablet for tablet b004b105a74e408380a4fe15bacbff03 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.923233 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b004b105a74e408380a4fe15bacbff03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.923787 27839 tablet_bootstrap.cc:492] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.923816 27127 master_service.cc:946] ReplaceTablet: received request to replace tablet 9ff52b758952450dbc6eb3162457000f from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.923830 27839 ts_tablet_manager.cc:1367] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:07.923952 27839 raft_consensus.cc:348] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.924012 27839 raft_consensus.cc:374] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.924031 27839 raft_consensus.cc:729] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.924064 27839 consensus_queue.cc:260] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.924113 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.924150 27839 ts_tablet_manager.cc:1398] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.924170 27844 tablet_replica.cc:330] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.924206 27844 raft_consensus.cc:2229] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.924207 27839 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.924233 27844 raft_consensus.cc:2258] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.924522 27839 tablet_bootstrap.cc:654] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.924692 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.924867 27123 master_service.cc:946] ReplaceTablet: received request to replace tablet ec323adbf97f48428c1e062aeadb9b88 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.924897 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.925297 27128 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 291c3bbf65b5464cbf8521617fba6365
I20240503 02:09:07.925422 27128 catalog_manager.cc:6554] ReplaceTablet: tablet 291c3bbf65b5464cbf8521617fba6365 deleted and replaced by tablet e43172b357ce45babdd7a4cfc2b8917a
I20240503 02:09:07.925384 28143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.925630 27844 ts_tablet_manager.cc:1875] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.925817 27126 master_service.cc:946] ReplaceTablet: received request to replace tablet e14a135e07464f7aaaf9179806eb5224 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:07.926079 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 9ff52b758952450dbc6eb3162457000f
I20240503 02:09:07.926158 27127 catalog_manager.cc:6554] ReplaceTablet: tablet 9ff52b758952450dbc6eb3162457000f deleted and replaced by tablet 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:07.926339 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff52b758952450dbc6eb3162457000f with delete_type TABLET_DATA_DELETED (replaced by tablet 8127ac16597e40fa9b1590a61430aea0) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.926401 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 291c3bbf65b5464cbf8521617fba6365 with delete_type TABLET_DATA_DELETED (replaced by tablet e43172b357ce45babdd7a4cfc2b8917a) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.926472 27845 tablet_replica.cc:330] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.926510 27845 raft_consensus.cc:2229] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.926537 27845 raft_consensus.cc:2258] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.926604 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 291c3bbf65b5464cbf8521617fba6365 with delete_type TABLET_DATA_DELETED (replaced by tablet e43172b357ce45babdd7a4cfc2b8917a) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.926658 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff52b758952450dbc6eb3162457000f with delete_type TABLET_DATA_DELETED (replaced by tablet 8127ac16597e40fa9b1590a61430aea0) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.926690 27843 tablet_replica.cc:330] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.926725 27845 ts_tablet_manager.cc:1875] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.926780 27843 raft_consensus.cc:2229] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.926821 27843 raft_consensus.cc:2258] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.926934 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet 291c3bbf65b5464cbf8521617fba6365 with delete_type TABLET_DATA_DELETED (replaced by tablet e43172b357ce45babdd7a4cfc2b8917a) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.927054 27844 ts_tablet_manager.cc:1888] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.927057 27843 ts_tablet_manager.cc:1875] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.927093 27844 log.cc:1196] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f5daed7b8f00446b8d94d71b1705ac01
I20240503 02:09:07.927229 27844 ts_tablet_manager.cc:1909] T f5daed7b8f00446b8d94d71b1705ac01 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.927358 27844 tablet_replica.cc:330] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.927390 27844 raft_consensus.cc:2229] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:07.927435 27844 raft_consensus.cc:2258] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.927440 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f5daed7b8f00446b8d94d71b1705ac01 (table default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]) successfully deleted
I20240503 02:09:07.927680 27844 ts_tablet_manager.cc:1875] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.927754 27845 ts_tablet_manager.cc:1888] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.927809 27845 log.cc:1196] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f5daed7b8f00446b8d94d71b1705ac01
I20240503 02:09:07.927961 27845 ts_tablet_manager.cc:1909] T f5daed7b8f00446b8d94d71b1705ac01 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.928162 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff52b758952450dbc6eb3162457000f with delete_type TABLET_DATA_DELETED (replaced by tablet 8127ac16597e40fa9b1590a61430aea0) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.928303 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet ec323adbf97f48428c1e062aeadb9b88
I20240503 02:09:07.928447 27845 tablet_replica.cc:330] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.928481 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f5daed7b8f00446b8d94d71b1705ac01 (table default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]) successfully deleted
I20240503 02:09:07.928498 27845 raft_consensus.cc:2229] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.928527 27845 raft_consensus.cc:2258] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.928556 27123 catalog_manager.cc:6554] ReplaceTablet: tablet ec323adbf97f48428c1e062aeadb9b88 deleted and replaced by tablet 2ad4f6e1ac014739b60a30111b64a7c1
I20240503 02:09:07.928731 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet ec323adbf97f48428c1e062aeadb9b88 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad4f6e1ac014739b60a30111b64a7c1) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.928890 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet ec323adbf97f48428c1e062aeadb9b88 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad4f6e1ac014739b60a30111b64a7c1) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.928942 27845 ts_tablet_manager.cc:1875] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.928937 28089 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:07.929026 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet ec323adbf97f48428c1e062aeadb9b88 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad4f6e1ac014739b60a30111b64a7c1) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.928745 27844 ts_tablet_manager.cc:1888] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.929155 27844 log.cc:1196] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9ff52b758952450dbc6eb3162457000f
I20240503 02:09:07.929005 28089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:07.929359 27844 ts_tablet_manager.cc:1909] T 9ff52b758952450dbc6eb3162457000f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.929356 28089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:07.929618 27844 tablet_replica.cc:330] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.929617 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9ff52b758952450dbc6eb3162457000f (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:07.929685 27844 raft_consensus.cc:2229] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.929721 27844 raft_consensus.cc:2258] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.929917 27844 ts_tablet_manager.cc:1875] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.930307 27845 ts_tablet_manager.cc:1888] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.930372 27845 log.cc:1196] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/291c3bbf65b5464cbf8521617fba6365
I20240503 02:09:07.930514 27845 ts_tablet_manager.cc:1909] T 291c3bbf65b5464cbf8521617fba6365 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.930846 27844 ts_tablet_manager.cc:1888] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.930896 27845 tablet_replica.cc:330] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.930936 27845 raft_consensus.cc:2229] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.930966 27845 raft_consensus.cc:2258] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.931066 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 291c3bbf65b5464cbf8521617fba6365 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:07.931119 27429 tablet_service.cc:1467] Processing CreateTablet for tablet e43172b357ce45babdd7a4cfc2b8917a (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.931155 27844 log.cc:1196] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/291c3bbf65b5464cbf8521617fba6365
I20240503 02:09:07.931243 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e43172b357ce45babdd7a4cfc2b8917a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.931371 27844 ts_tablet_manager.cc:1909] T 291c3bbf65b5464cbf8521617fba6365 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.931391 27564 tablet_service.cc:1467] Processing CreateTablet for tablet e43172b357ce45babdd7a4cfc2b8917a (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.931488 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e43172b357ce45babdd7a4cfc2b8917a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.931579 27700 tablet_service.cc:1467] Processing CreateTablet for tablet e43172b357ce45babdd7a4cfc2b8917a (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.931650 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e43172b357ce45babdd7a4cfc2b8917a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.931937 27843 ts_tablet_manager.cc:1888] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.931982 27843 log.cc:1196] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/291c3bbf65b5464cbf8521617fba6365
I20240503 02:09:07.932126 27843 ts_tablet_manager.cc:1909] T 291c3bbf65b5464cbf8521617fba6365 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.932348 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 291c3bbf65b5464cbf8521617fba6365 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:07.932734 27128 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.933009 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.933094 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad4f6e1ac014739b60a30111b64a7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.933274 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 8127ac16597e40fa9b1590a61430aea0 (DEFAULT_TABLE table=default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.933301 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.933383 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8127ac16597e40fa9b1590a61430aea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.933542 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad4f6e1ac014739b60a30111b64a7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.934370 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 (DEFAULT_TABLE table=default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.934451 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ad4f6e1ac014739b60a30111b64a7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.934669 27844 tablet_replica.cc:330] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.934677 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 291c3bbf65b5464cbf8521617fba6365 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:07.934754 27844 raft_consensus.cc:2229] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.934899 27843 tablet_replica.cc:330] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.934814 27844 raft_consensus.cc:2258] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.935271 27844 ts_tablet_manager.cc:1875] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.935323 27126 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet e14a135e07464f7aaaf9179806eb5224
I20240503 02:09:07.935420 27126 catalog_manager.cc:6554] ReplaceTablet: tablet e14a135e07464f7aaaf9179806eb5224 deleted and replaced by tablet 4ea3f123adc34f7db81d534fdb78c505
I20240503 02:09:07.935487 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 8127ac16597e40fa9b1590a61430aea0 (DEFAULT_TABLE table=default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.935556 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet e14a135e07464f7aaaf9179806eb5224 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea3f123adc34f7db81d534fdb78c505) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:07.935631 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8127ac16597e40fa9b1590a61430aea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.935650 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet e14a135e07464f7aaaf9179806eb5224 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea3f123adc34f7db81d534fdb78c505) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:07.935046 27845 ts_tablet_manager.cc:1875] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.936093 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet e14a135e07464f7aaaf9179806eb5224 with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea3f123adc34f7db81d534fdb78c505) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.936580 27843 raft_consensus.cc:2229] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.937067 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.937148 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea3f123adc34f7db81d534fdb78c505. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.937371 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 8127ac16597e40fa9b1590a61430aea0 (DEFAULT_TABLE table=default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.937459 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8127ac16597e40fa9b1590a61430aea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.937448 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.937536 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea3f123adc34f7db81d534fdb78c505. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.938335 27845 ts_tablet_manager.cc:1888] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.936749 27843 raft_consensus.cc:2258] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.937443 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 (DEFAULT_TABLE table=default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:07.938624 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ea3f123adc34f7db81d534fdb78c505. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:07.938753 27843 ts_tablet_manager.cc:1875] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.938925 27844 ts_tablet_manager.cc:1888] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.938966 27844 log.cc:1196] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ec323adbf97f48428c1e062aeadb9b88
I20240503 02:09:07.939098 27844 ts_tablet_manager.cc:1909] T ec323adbf97f48428c1e062aeadb9b88 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.939324 27838 tablet_bootstrap.cc:492] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.939359 27838 ts_tablet_manager.cc:1367] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.000s
I20240503 02:09:07.939469 27838 raft_consensus.cc:348] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.939529 27838 raft_consensus.cc:374] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.939571 27838 raft_consensus.cc:729] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.939610 27838 consensus_queue.cc:260] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.939633 27845 log.cc:1196] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9ff52b758952450dbc6eb3162457000f
I20240503 02:09:07.939677 27838 ts_tablet_manager.cc:1398] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.939755 27838 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.940047 27838 tablet_bootstrap.cc:654] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.940035 27843 ts_tablet_manager.cc:1888] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:07.940125 27843 log.cc:1196] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9ff52b758952450dbc6eb3162457000f
I20240503 02:09:07.940239 27845 ts_tablet_manager.cc:1909] T 9ff52b758952450dbc6eb3162457000f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.940356 27843 ts_tablet_manager.cc:1909] T 9ff52b758952450dbc6eb3162457000f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.940526 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9ff52b758952450dbc6eb3162457000f (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:07.940599 27845 tablet_replica.cc:330] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.940636 27845 raft_consensus.cc:2229] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.940661 27845 raft_consensus.cc:2258] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.940783 27839 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.940822 27839 ts_tablet_manager.cc:1367] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.000s
I20240503 02:09:07.940938 27839 raft_consensus.cc:348] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.940999 27839 raft_consensus.cc:374] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.941016 27839 raft_consensus.cc:729] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.941059 27839 consensus_queue.cc:260] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.941082 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet f5daed7b8f00446b8d94d71b1705ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:07 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.941126 27844 tablet_replica.cc:330] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:07.941131 27839 ts_tablet_manager.cc:1398] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.941150 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ec323adbf97f48428c1e062aeadb9b88 (table default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:07.941167 27844 raft_consensus.cc:2229] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.941186 27839 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.941197 27844 raft_consensus.cc:2258] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.941233 27845 ts_tablet_manager.cc:1875] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.941381 27844 ts_tablet_manager.cc:1875] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.941495 27839 tablet_bootstrap.cc:654] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.942307 27843 tablet_replica.cc:330] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.942374 27843 raft_consensus.cc:2229] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.942420 27843 raft_consensus.cc:2258] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.942550 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9ff52b758952450dbc6eb3162457000f (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
W20240503 02:09:07.942617 27117 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet ec323adbf97f48428c1e062aeadb9b88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec323adbf97f48428c1e062aeadb9b88 already in progress: creating tablet
I20240503 02:09:07.942693 27843 ts_tablet_manager.cc:1875] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.942930 27845 ts_tablet_manager.cc:1888] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.943019 27845 log.cc:1196] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ec323adbf97f48428c1e062aeadb9b88
I20240503 02:09:07.943194 27845 ts_tablet_manager.cc:1909] T ec323adbf97f48428c1e062aeadb9b88 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.943218 27844 ts_tablet_manager.cc:1888] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.943274 27844 log.cc:1196] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e14a135e07464f7aaaf9179806eb5224
I20240503 02:09:07.943421 27844 ts_tablet_manager.cc:1909] T e14a135e07464f7aaaf9179806eb5224 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:07.943511 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ec323adbf97f48428c1e062aeadb9b88 (table default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:07.943607 27845 tablet_replica.cc:330] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:07.943672 27845 raft_consensus.cc:2229] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.943701 27843 ts_tablet_manager.cc:1888] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.943711 27837 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.943727 27845 raft_consensus.cc:2258] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.943747 27837 ts_tablet_manager.cc:1367] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20240503 02:09:07.943797 27843 log.cc:1196] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e14a135e07464f7aaaf9179806eb5224
I20240503 02:09:07.943864 27837 raft_consensus.cc:348] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.943923 27837 raft_consensus.cc:374] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.943951 27837 raft_consensus.cc:729] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.943920 27845 ts_tablet_manager.cc:1875] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.943989 27837 consensus_queue.cc:260] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.944031 27843 ts_tablet_manager.cc:1909] T e14a135e07464f7aaaf9179806eb5224 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.944069 27837 ts_tablet_manager.cc:1398] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.944133 27837 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.944445 27843 tablet_replica.cc:330] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.944519 27837 tablet_bootstrap.cc:654] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.944564 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e14a135e07464f7aaaf9179806eb5224 (table default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:07.944574 27843 raft_consensus.cc:2229] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.944633 27843 raft_consensus.cc:2258] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.945119 27837 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.945160 27837 ts_tablet_manager.cc:1367] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.945223 27843 ts_tablet_manager.cc:1875] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.945266 27837 raft_consensus.cc:348] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.945314 27837 raft_consensus.cc:374] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.945369 27837 raft_consensus.cc:729] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.945402 27837 consensus_queue.cc:260] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.945474 27837 ts_tablet_manager.cc:1398] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.945533 27837 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.945833 27837 tablet_bootstrap.cc:654] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.946852 27843 ts_tablet_manager.cc:1888] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.946930 27837 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.946965 27837 ts_tablet_manager.cc:1367] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.946972 27845 ts_tablet_manager.cc:1888] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.947017 27845 log.cc:1196] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e14a135e07464f7aaaf9179806eb5224
I20240503 02:09:07.947074 27843 log.cc:1196] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f5daed7b8f00446b8d94d71b1705ac01
I20240503 02:09:07.947094 27837 raft_consensus.cc:348] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.947150 27837 raft_consensus.cc:374] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.947171 27837 raft_consensus.cc:729] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.947216 27837 consensus_queue.cc:260] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.947307 27845 ts_tablet_manager.cc:1909] T e14a135e07464f7aaaf9179806eb5224 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:07.947314 27837 ts_tablet_manager.cc:1398] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.947376 27837 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.947564 27843 ts_tablet_manager.cc:1909] T f5daed7b8f00446b8d94d71b1705ac01 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.947680 27837 tablet_bootstrap.cc:654] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.947852 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f5daed7b8f00446b8d94d71b1705ac01 (table default.Table_b5b9dfae61664236992fba8ef989ac69 [id=2ea9fcf5fc494b54b2250fe70dfd5810]) successfully deleted
I20240503 02:09:07.948130 27838 tablet_bootstrap.cc:492] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.948171 27838 ts_tablet_manager.cc:1367] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:07.948256 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e14a135e07464f7aaaf9179806eb5224 (table default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:07.948297 27839 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.948292 27838 raft_consensus.cc:348] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.948334 27839 ts_tablet_manager.cc:1367] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:07.948340 27838 raft_consensus.cc:374] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.948359 27838 raft_consensus.cc:729] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.948395 27838 consensus_queue.cc:260] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.948454 27839 raft_consensus.cc:348] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.948490 27838 ts_tablet_manager.cc:1398] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.948500 27839 raft_consensus.cc:374] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.948518 27839 raft_consensus.cc:729] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.948558 27838 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.948554 27839 consensus_queue.cc:260] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.948621 27839 ts_tablet_manager.cc:1398] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.948678 27839 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.948897 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e14a135e07464f7aaaf9179806eb5224 (table default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:07.948987 27838 tablet_bootstrap.cc:654] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.948993 27839 tablet_bootstrap.cc:654] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.950508 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet ec323adbf97f48428c1e062aeadb9b88 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad4f6e1ac014739b60a30111b64a7c1) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:07.950564 27837 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.950584 27843 tablet_replica.cc:330] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:07.950608 27837 ts_tablet_manager.cc:1367] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.950624 27843 raft_consensus.cc:2229] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:07.950690 27843 raft_consensus.cc:2258] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:07.950721 27837 raft_consensus.cc:348] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.950778 27837 raft_consensus.cc:374] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.950799 27837 raft_consensus.cc:729] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.950834 27837 consensus_queue.cc:260] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.950881 27838 tablet_bootstrap.cc:492] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.950917 27838 ts_tablet_manager.cc:1367] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.950918 27837 ts_tablet_manager.cc:1398] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.950978 27837 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.951028 27838 raft_consensus.cc:348] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.951084 27838 raft_consensus.cc:374] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.951099 27838 raft_consensus.cc:729] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.951124 27838 consensus_queue.cc:260] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.951184 27838 ts_tablet_manager.cc:1398] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.951244 27838 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.951273 27837 tablet_bootstrap.cc:654] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.951550 27838 tablet_bootstrap.cc:654] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.951628 27843 ts_tablet_manager.cc:1875] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:07.952438 27837 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.952490 27837 ts_tablet_manager.cc:1367] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:07.952611 27837 raft_consensus.cc:348] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.952669 27837 raft_consensus.cc:374] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.952692 27837 raft_consensus.cc:729] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.952723 27837 consensus_queue.cc:260] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.952811 27837 ts_tablet_manager.cc:1398] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:07.952868 27837 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:07.953131 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.953176 27837 tablet_bootstrap.cc:654] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:07.953219 27126 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
I20240503 02:09:07.953852 27839 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.953896 27839 ts_tablet_manager.cc:1367] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:07.953969 27838 tablet_bootstrap.cc:492] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.954001 27838 ts_tablet_manager.cc:1367] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:07.954042 27839 raft_consensus.cc:348] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.954106 27839 raft_consensus.cc:374] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.954133 27839 raft_consensus.cc:729] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.954120 27838 raft_consensus.cc:348] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.954177 27838 raft_consensus.cc:374] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.954175 27839 consensus_queue.cc:260] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.954205 27838 raft_consensus.cc:729] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.954248 27838 consensus_queue.cc:260] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.954270 27839 ts_tablet_manager.cc:1398] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.954326 27839 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.954324 27838 ts_tablet_manager.cc:1398] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.954386 27838 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.954392 27126 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.954656 27839 tablet_bootstrap.cc:654] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.954715 27838 tablet_bootstrap.cc:654] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.955578 27837 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:07.955621 27837 ts_tablet_manager.cc:1367] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:07.955670 27843 ts_tablet_manager.cc:1888] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:07.955736 27843 log.cc:1196] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ec323adbf97f48428c1e062aeadb9b88
I20240503 02:09:07.955736 27837 raft_consensus.cc:348] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.955787 27837 raft_consensus.cc:374] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.955803 27837 raft_consensus.cc:729] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.955797 28089 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:07.955850 27837 consensus_queue.cc:260] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.955935 27837 ts_tablet_manager.cc:1398] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.955991 27843 ts_tablet_manager.cc:1909] T ec323adbf97f48428c1e062aeadb9b88 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:07.956244 28089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:07.956305 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ec323adbf97f48428c1e062aeadb9b88 (table default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:07.956398 28089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:07.956413 27838 tablet_bootstrap.cc:492] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.956449 27838 ts_tablet_manager.cc:1367] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:07.956568 27838 raft_consensus.cc:348] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.956625 27838 raft_consensus.cc:374] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.956651 27838 raft_consensus.cc:729] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.956691 27838 consensus_queue.cc:260] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.956770 27838 ts_tablet_manager.cc:1398] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.956836 27838 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.957132 27838 tablet_bootstrap.cc:654] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.957772 27839 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.957839 27839 ts_tablet_manager.cc:1367] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:07.957960 27839 raft_consensus.cc:348] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.958031 27839 raft_consensus.cc:374] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.958058 27839 raft_consensus.cc:729] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.958101 27839 consensus_queue.cc:260] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.958181 27839 ts_tablet_manager.cc:1398] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.958232 27839 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.958273 27838 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.958310 27838 ts_tablet_manager.cc:1367] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.958428 27838 raft_consensus.cc:348] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.958489 27838 raft_consensus.cc:374] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.958508 27838 raft_consensus.cc:729] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.958536 27838 consensus_queue.cc:260] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.958611 27838 ts_tablet_manager.cc:1398] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.958642 27839 tablet_bootstrap.cc:654] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.958667 27838 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.958963 27838 tablet_bootstrap.cc:654] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.958988 27904 raft_consensus.cc:482] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.959038 27904 raft_consensus.cc:504] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.959154 27904 leader_election.cc:288] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.959161 27839 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.959197 27839 ts_tablet_manager.cc:1367] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.959308 27839 raft_consensus.cc:348] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.959370 27839 raft_consensus.cc:374] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.959388 27839 raft_consensus.cc:729] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.959306 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccd6dc9cfb834653ae08c29ff0f4848d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.959439 27839 consensus_queue.cc:260] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.959494 27451 raft_consensus.cc:2454] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.959563 27839 ts_tablet_manager.cc:1398] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.959619 27839 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:07.959717 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccd6dc9cfb834653ae08c29ff0f4848d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.959780 27588 raft_consensus.cc:2454] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:07.960011 27838 tablet_bootstrap.cc:492] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.960052 27838 ts_tablet_manager.cc:1367] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:07.960085 27839 tablet_bootstrap.cc:654] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.960170 27838 raft_consensus.cc:348] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.960218 27656 leader_election.cc:302] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.960330 27853 raft_consensus.cc:2781] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.960363 27853 raft_consensus.cc:482] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.960381 27853 raft_consensus.cc:3037] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.960822 28089 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.60:36369
I20240503 02:09:07.960929 27853 raft_consensus.cc:504] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.961030 27853 leader_election.cc:288] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.961112 27128 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.961179 28089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
I20240503 02:09:07.961448 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccd6dc9cfb834653ae08c29ff0f4848d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.961522 27451 raft_consensus.cc:3037] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.961648 28212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.60:36369 every 8 connection(s)
I20240503 02:09:07.961813 27848 raft_consensus.cc:482] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.961855 27848 raft_consensus.cc:504] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.961942 27848 leader_election.cc:288] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.962086 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e1f031e13f47c692de9a8a5e256613" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.962102 27451 raft_consensus.cc:2454] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.962137 27453 raft_consensus.cc:2454] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.962343 27659 leader_election.cc:302] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.962443 27524 leader_election.cc:302] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.960220 27838 raft_consensus.cc:374] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.962495 27838 raft_consensus.cc:729] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.962481 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e1f031e13f47c692de9a8a5e256613" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.962520 27848 raft_consensus.cc:2781] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.962532 27722 raft_consensus.cc:2454] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.962543 27848 raft_consensus.cc:482] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.962565 27848 raft_consensus.cc:3037] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.962538 27838 consensus_queue.cc:260] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.962671 28213 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:09:07.962682 27838 ts_tablet_manager.cc:1398] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:07.962746 27838 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:07.963070 27838 tablet_bootstrap.cc:654] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:07.963126 27848 raft_consensus.cc:504] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.963236 27848 leader_election.cc:288] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.962445 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccd6dc9cfb834653ae08c29ff0f4848d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.963307 27588 raft_consensus.cc:3037] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.963359 28213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap starting.
I20240503 02:09:07.963599 27853 raft_consensus.cc:2781] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.963646 27853 raft_consensus.cc:686] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:07.963680 27853 consensus_queue.cc:237] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.963855 27588 raft_consensus.cc:2454] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:07.964241 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e1f031e13f47c692de9a8a5e256613" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.964286 27839 tablet_bootstrap.cc:492] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:07.964300 27722 raft_consensus.cc:3037] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.964318 27839 ts_tablet_manager.cc:1367] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:07.964427 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e1f031e13f47c692de9a8a5e256613" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.964493 27453 raft_consensus.cc:3037] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.964771 28213 log.cc:824] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:07.964869 27722 raft_consensus.cc:2454] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.965056 27453 raft_consensus.cc:2454] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.965201 27523 leader_election.cc:302] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.964432 27839 raft_consensus.cc:348] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.965245 27839 raft_consensus.cc:374] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.965263 27839 raft_consensus.cc:729] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.965296 27848 raft_consensus.cc:2781] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.965299 27839 consensus_queue.cc:260] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.965438 27839 ts_tablet_manager.cc:1398] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:07.965584 27848 raft_consensus.cc:686] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.965636 27848 consensus_queue.cc:237] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.966251 27128 catalog_manager.cc:5484] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.966358 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_c456a5e68e2f4baaaec502f69453d04b [id=453d4c39809a42f38267db3c4d45748a]). Expected version 1 got 0
I20240503 02:09:07.966399 28213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20240503 02:09:07.966531 28054 consensus_queue.cc:1035] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.966638 28213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap complete.
I20240503 02:09:07.966635 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 28089
I20240503 02:09:07.966719 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.967250 27123 catalog_manager.cc:5484] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.967671 28007 raft_consensus.cc:482] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.967712 28007 raft_consensus.cc:504] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.967800 28007 leader_election.cc:288] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.967962 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da16da72302e49568ea8d2967f0d2e7a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.967993 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446]). Expected version 1 got 0
I20240503 02:09:07.968021 27453 raft_consensus.cc:2454] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.968070 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da16da72302e49568ea8d2967f0d2e7a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.968129 27722 raft_consensus.cc:2454] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.968132 27524 leader_election.cc:302] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.968231 28007 raft_consensus.cc:2781] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.968256 28007 raft_consensus.cc:482] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.968272 28007 raft_consensus.cc:3037] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.968855 28007 raft_consensus.cc:504] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.968950 28007 leader_election.cc:288] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.968600 28213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:07.969580 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da16da72302e49568ea8d2967f0d2e7a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.969652 27453 raft_consensus.cc:3037] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.969784 27838 tablet_bootstrap.cc:492] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:07.969820 27838 ts_tablet_manager.cc:1367] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:07.969935 27838 raft_consensus.cc:348] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.969980 27838 raft_consensus.cc:374] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:07.969996 27838 raft_consensus.cc:729] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.970031 27838 consensus_queue.cc:260] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.970100 27838 ts_tablet_manager.cc:1398] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:07.970175 27453 raft_consensus.cc:2454] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.970243 28213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c75a5f0d0824fafbc14d2a2c1613db1, State: Initialized, Role: FOLLOWER
I20240503 02:09:07.970368 28213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:07.970423 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da16da72302e49568ea8d2967f0d2e7a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.970474 27722 raft_consensus.cc:3037] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.970494 27524 leader_election.cc:302] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.970639 28213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:07.971009 27722 raft_consensus.cc:2454] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:07.971072 28007 raft_consensus.cc:2781] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.971120 28007 raft_consensus.cc:686] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:07.971154 28007 consensus_queue.cc:237] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:07.971971 27123 catalog_manager.cc:5484] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:07.972069 27123 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.973078 27852 consensus_queue.cc:1035] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.973315 28238 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:07.973606 28238 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: success
I20240503 02:09:07.974738 28238 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: acquiring CA information for follower catalog manager: success
I20240503 02:09:07.974993 27499 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:07.975039 27499 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.975222 28238 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:07.975916 28173 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" instance_seqno: 1714702146630581) as {username='slave'} at 127.26.117.1:41401; Asking this server to re-register.
I20240503 02:09:07.976240 27499 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:07.976639 28173 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:07.977206 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:07.977447 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:07.978309 27634 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:07.978356 27634 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.978344 27453 raft_consensus.cc:1261] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.978500 27722 raft_consensus.cc:1261] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.978505 28007 consensus_queue.cc:1035] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.978688 28007 consensus_queue.cc:1035] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.979653 27852 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.979676 27846 raft_consensus.cc:482] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.979682 27126 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.979702 27852 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.979705 27846 raft_consensus.cc:504] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.979779 27123 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:07.979799 27852 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.979815 27846 leader_election.cc:288] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.979965 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203665395f3745f196fb393dd82b8fc9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.979976 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203665395f3745f196fb393dd82b8fc9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.979974 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.980031 27721 raft_consensus.cc:2454] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.980031 27588 raft_consensus.cc:2454] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.980059 27587 raft_consensus.cc:2454] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.980086 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.980129 27723 raft_consensus.cc:2454] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.980250 27386 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.980321 27386 leader_election.cc:302] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.980358 28173 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" instance_seqno: 1714702146750084) as {username='slave'} at 127.26.117.2:52023; Asking this server to re-register.
I20240503 02:09:07.980393 27852 raft_consensus.cc:2781] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.980423 27852 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.980440 27852 raft_consensus.cc:3037] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.980556 28007 raft_consensus.cc:482] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.980598 28007 raft_consensus.cc:504] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.980690 28007 leader_election.cc:288] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.980778 27846 raft_consensus.cc:2781] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.980813 27846 raft_consensus.cc:482] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.980829 27846 raft_consensus.cc:3037] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.981020 27852 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.981071 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.981118 27723 raft_consensus.cc:2454] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:07.981128 27852 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.981168 27769 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:07.981199 27769 heartbeater.cc:503] Master 127.26.117.60:36369 was elected leader, sending a full tablet report...
I20240503 02:09:07.981230 27523 leader_election.cc:302] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.981384 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:07.981451 27846 raft_consensus.cc:504] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.981551 27846 leader_election.cc:288] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.981595 27451 raft_consensus.cc:2379] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:07.981716 28173 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" instance_seqno: 1714702146863903) as {username='slave'} at 127.26.117.3:44917; Asking this server to re-register.
I20240503 02:09:07.981775 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.981828 27721 raft_consensus.cc:3037] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.981891 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203665395f3745f196fb393dd82b8fc9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.981922 27634 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:07.981943 27723 raft_consensus.cc:3037] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.982133 27769 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:07.982437 27721 raft_consensus.cc:2454] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.982501 27723 raft_consensus.cc:2454] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.982532 28007 raft_consensus.cc:2781] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.982564 28007 raft_consensus.cc:482] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.982582 28007 raft_consensus.cc:3037] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.982729 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.982775 28173 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.982839 28174 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:07.982892 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "203665395f3745f196fb393dd82b8fc9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.982939 27587 raft_consensus.cc:3037] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.983121 28007 raft_consensus.cc:504] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.983220 27588 raft_consensus.cc:2379] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:07.983237 28007 leader_election.cc:288] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.983410 27388 leader_election.cc:302] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.983490 27388 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:07.983515 27587 raft_consensus.cc:2454] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.983516 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:07.983592 27451 raft_consensus.cc:2379] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:07.983686 27852 raft_consensus.cc:2781] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.983665 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16577f3067c04965887c8b3db12795e4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.983726 27723 raft_consensus.cc:2379] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:07.983724 27852 raft_consensus.cc:686] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.983738 28054 raft_consensus.cc:2781] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.983762 27852 consensus_queue.cc:237] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.983844 28054 raft_consensus.cc:686] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.983882 28054 consensus_queue.cc:237] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.984448 27523 leader_election.cc:302] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:07.984431 27123 catalog_manager.cc:5484] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.984529 27123 catalog_manager.cc:5484] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:07.984553 28007 raft_consensus.cc:2726] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:09:07.986805 27917 tablet.cc:1722] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.987236 27893 tablet.cc:1722] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.988319 27123 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd] alter complete (version 1)
I20240503 02:09:07.990327 27892 tablet.cc:1722] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.992269 27453 raft_consensus.cc:1261] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.992269 27722 raft_consensus.cc:1261] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:07.992621 27848 consensus_queue.cc:1035] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.992782 27848 consensus_queue.cc:1035] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:07.993958 27123 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_c456a5e68e2f4baaaec502f69453d04b [id=453d4c39809a42f38267db3c4d45748a]). Expected version 1 got 0
I20240503 02:09:07.994042 27126 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_c456a5e68e2f4baaaec502f69453d04b [id=453d4c39809a42f38267db3c4d45748a]). Expected version 1 got 0
I20240503 02:09:07.994364 27917 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.994560 27912 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.995968 27893 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:07.996248 27128 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_c456a5e68e2f4baaaec502f69453d04b [id=453d4c39809a42f38267db3c4d45748a]). Expected version 1 got 0
I20240503 02:09:07.996424 27846 raft_consensus.cc:482] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:07.996482 27846 raft_consensus.cc:504] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.996583 27846 leader_election.cc:288] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.996742 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b004b105a74e408380a4fe15bacbff03" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:07.996806 27587 raft_consensus.cc:2454] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.996871 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b004b105a74e408380a4fe15bacbff03" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:07.996922 27722 raft_consensus.cc:2454] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:07.996908 27386 leader_election.cc:302] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.996990 27846 raft_consensus.cc:2781] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:07.996999 27123 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_c456a5e68e2f4baaaec502f69453d04b [id=453d4c39809a42f38267db3c4d45748a] alter complete (version 1)
I20240503 02:09:07.997025 27846 raft_consensus.cc:482] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:07.997042 27846 raft_consensus.cc:3037] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.997617 27846 raft_consensus.cc:504] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:07.997720 27846 leader_election.cc:288] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:07.997965 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b004b105a74e408380a4fe15bacbff03" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:07.998023 27587 raft_consensus.cc:3037] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.998551 27587 raft_consensus.cc:2454] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.998759 27386 leader_election.cc:302] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:07.998869 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b004b105a74e408380a4fe15bacbff03" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:07.998927 27722 raft_consensus.cc:3037] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:07.999367 27846 raft_consensus.cc:2781] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:07.999410 27846 raft_consensus.cc:686] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:07.999444 27722 raft_consensus.cc:2454] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:07.999451 27846 consensus_queue.cc:237] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.001579 27126 catalog_manager.cc:5484] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.008826 28054 raft_consensus.cc:482] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.008883 28054 raft_consensus.cc:504] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.008991 28054 leader_election.cc:288] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.009119 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea3f123adc34f7db81d534fdb78c505" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.009182 27587 raft_consensus.cc:2454] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.009199 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea3f123adc34f7db81d534fdb78c505" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.009246 27722 raft_consensus.cc:2454] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.009310 27386 leader_election.cc:302] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.009402 28054 raft_consensus.cc:2781] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.009447 28054 raft_consensus.cc:482] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.009472 28054 raft_consensus.cc:3037] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.010002 28054 raft_consensus.cc:504] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.010121 28054 leader_election.cc:288] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.010227 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea3f123adc34f7db81d534fdb78c505" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.010277 27587 raft_consensus.cc:3037] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.010366 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ea3f123adc34f7db81d534fdb78c505" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.010421 27722 raft_consensus.cc:3037] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.010661 27587 raft_consensus.cc:2454] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.010921 27722 raft_consensus.cc:2454] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.011077 27386 leader_election.cc:302] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.011166 28054 raft_consensus.cc:2781] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.011227 28054 raft_consensus.cc:686] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.011268 28054 consensus_queue.cc:237] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.011960 27126 catalog_manager.cc:5484] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.016289 27900 raft_consensus.cc:482] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.016345 27900 raft_consensus.cc:504] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.016449 27900 leader_election.cc:288] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.016551 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442400993f8e43b79e429758d1f3c72f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.016564 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442400993f8e43b79e429758d1f3c72f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.016610 27453 raft_consensus.cc:2454] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.016615 27587 raft_consensus.cc:2454] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.016716 27659 leader_election.cc:302] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.016795 27900 raft_consensus.cc:2781] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.016824 27900 raft_consensus.cc:482] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.016841 27900 raft_consensus.cc:3037] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.017480 27900 raft_consensus.cc:504] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.017585 27900 leader_election.cc:288] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.017699 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442400993f8e43b79e429758d1f3c72f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.017753 27587 raft_consensus.cc:3037] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.017731 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "442400993f8e43b79e429758d1f3c72f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.017781 27453 raft_consensus.cc:3037] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.018311 27453 raft_consensus.cc:2454] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.018316 27587 raft_consensus.cc:2454] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.018466 27659 leader_election.cc:302] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.018535 27848 raft_consensus.cc:482] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.018570 27848 raft_consensus.cc:504] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.018667 27848 leader_election.cc:288] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.018787 27900 raft_consensus.cc:2781] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.018792 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e353654920a44994ae4889144db1b836" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.018832 27900 raft_consensus.cc:686] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.018847 27453 raft_consensus.cc:2454] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.018874 27900 consensus_queue.cc:237] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.018999 27524 leader_election.cc:302] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.019170 27848 raft_consensus.cc:2781] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.019209 27848 raft_consensus.cc:482] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.019227 27848 raft_consensus.cc:3037] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.019703 27848 raft_consensus.cc:504] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.019738 28054 raft_consensus.cc:482] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.019807 27848 leader_election.cc:288] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.019796 28054 raft_consensus.cc:504] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.019881 28054 leader_election.cc:288] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.019948 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e353654920a44994ae4889144db1b836" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.020004 27453 raft_consensus.cc:3037] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.020022 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e353654920a44994ae4889144db1b836" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.020076 27722 raft_consensus.cc:2454] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.020516 27453 raft_consensus.cc:2454] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.020653 27126 catalog_manager.cc:5484] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.020750 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e353654920a44994ae4889144db1b836" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.020798 27723 raft_consensus.cc:3037] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.020778 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad4f6e1ac014739b60a30111b64a7c1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.020834 27722 raft_consensus.cc:2454] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.020912 27524 leader_election.cc:302] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.021085 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad4f6e1ac014739b60a30111b64a7c1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.021132 27587 raft_consensus.cc:2454] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.021260 27884 raft_consensus.cc:2781] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.021306 27884 raft_consensus.cc:686] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:08.021344 27723 raft_consensus.cc:2454] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.021370 27884 consensus_queue.cc:237] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.021526 27884 raft_consensus.cc:482] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.021555 27884 raft_consensus.cc:504] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.021629 27884 leader_election.cc:288] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.021744 27386 leader_election.cc:302] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.021983 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89440f04f9d349a187c296efab0816d2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.021970 27128 catalog_manager.cc:5484] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.022043 27723 raft_consensus.cc:2454] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.022318 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89440f04f9d349a187c296efab0816d2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.022388 27453 raft_consensus.cc:2454] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.022462 27523 leader_election.cc:302] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.022578 28007 raft_consensus.cc:2781] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.022584 28054 raft_consensus.cc:2781] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.022609 28007 raft_consensus.cc:482] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.022615 28054 raft_consensus.cc:482] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.022627 28007 raft_consensus.cc:3037] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.022634 28054 raft_consensus.cc:3037] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.023002 28007 raft_consensus.cc:504] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.023079 28007 leader_election.cc:288] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.023144 28054 raft_consensus.cc:504] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.023224 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89440f04f9d349a187c296efab0816d2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.023243 28054 leader_election.cc:288] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.023267 27723 raft_consensus.cc:3037] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.023509 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad4f6e1ac014739b60a30111b64a7c1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.023562 27722 raft_consensus.cc:3037] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.023743 27723 raft_consensus.cc:2454] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.023862 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ad4f6e1ac014739b60a30111b64a7c1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.023918 27587 raft_consensus.cc:3037] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.023993 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89440f04f9d349a187c296efab0816d2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.024039 27453 raft_consensus.cc:3037] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.024075 27722 raft_consensus.cc:2454] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.024406 27587 raft_consensus.cc:2454] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.024497 27453 raft_consensus.cc:2454] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.024561 27388 leader_election.cc:302] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.024647 27524 leader_election.cc:302] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.024722 28007 raft_consensus.cc:2781] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.024765 28007 raft_consensus.cc:686] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:08.024809 28007 consensus_queue.cc:237] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.024893 28054 raft_consensus.cc:2781] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.024955 28054 raft_consensus.cc:686] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.024987 28054 consensus_queue.cc:237] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:08.025558 27123 catalog_manager.cc:5484] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.026989 27128 catalog_manager.cc:5484] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.027082 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.029899 27846 raft_consensus.cc:482] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.029950 27846 raft_consensus.cc:504] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.030045 27846 leader_election.cc:288] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.030154 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43172b357ce45babdd7a4cfc2b8917a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.030156 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43172b357ce45babdd7a4cfc2b8917a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.030212 27722 raft_consensus.cc:2454] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.030212 27587 raft_consensus.cc:2454] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.030319 27388 leader_election.cc:302] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.030400 27846 raft_consensus.cc:2781] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.030426 27846 raft_consensus.cc:482] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.030447 27846 raft_consensus.cc:3037] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.030894 27846 raft_consensus.cc:504] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.031008 27846 leader_election.cc:288] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.031133 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43172b357ce45babdd7a4cfc2b8917a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.031194 27722 raft_consensus.cc:3037] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.031666 27722 raft_consensus.cc:2454] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.031744 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43172b357ce45babdd7a4cfc2b8917a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.031790 27587 raft_consensus.cc:3037] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.031965 27388 leader_election.cc:302] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.032054 27846 raft_consensus.cc:2781] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.032101 27846 raft_consensus.cc:686] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.032135 27846 consensus_queue.cc:237] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.032239 27587 raft_consensus.cc:2454] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.032732 27128 catalog_manager.cc:5484] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.042752 27128 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_7310e9433672464690114ff02b5c773f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.042851 27128 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7310e9433672464690114ff02b5c773f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.051196 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_177f76165820429abc622c2444923017"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:08.051226 27122 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_c333427c445048efb408340ce1cdef66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.051322 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_177f76165820429abc622c2444923017 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.051239 27127 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_b795b4876be143edb345a11ccda4a55b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:08.051196 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_9fc74e464bc14c979b42aa031fc368f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.051416 27127 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b795b4876be143edb345a11ccda4a55b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:08.051452 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fc74e464bc14c979b42aa031fc368f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:08.051343 27122 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c333427c445048efb408340ce1cdef66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.054432 27701 tablet_service.cc:1467] Processing CreateTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 (DEFAULT_TABLE table=default.Table_177f76165820429abc622c2444923017 [id=846642e389d8455184d0ce48466fcbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.054548 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1fd17aa4a24b3a903c71482333ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.054643 27429 tablet_service.cc:1467] Processing CreateTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 (DEFAULT_TABLE table=default.Table_177f76165820429abc622c2444923017 [id=846642e389d8455184d0ce48466fcbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.054736 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1fd17aa4a24b3a903c71482333ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.055393 27838 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.055447 27425 tablet_service.cc:1467] Processing CreateTablet for tablet be7a66cc87834e739e958a2caf73de0c (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.055406 27700 tablet_service.cc:1467] Processing CreateTablet for tablet be7a66cc87834e739e958a2caf73de0c (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.055539 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7a66cc87834e739e958a2caf73de0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.055552 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7a66cc87834e739e958a2caf73de0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.055586 27839 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.055722 27838 tablet_bootstrap.cc:654] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.055893 27839 tablet_bootstrap.cc:654] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.056545 27839 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.056584 27839 ts_tablet_manager.cc:1367] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.056689 27563 tablet_service.cc:1467] Processing CreateTablet for tablet be7a66cc87834e739e958a2caf73de0c (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.056710 27839 raft_consensus.cc:348] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.056761 27839 raft_consensus.cc:374] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.056773 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7a66cc87834e739e958a2caf73de0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.056777 27839 raft_consensus.cc:729] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.056824 27839 consensus_queue.cc:260] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.056912 27839 ts_tablet_manager.cc:1398] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.056955 27839 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.057260 27839 tablet_bootstrap.cc:654] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.057627 27564 tablet_service.cc:1467] Processing CreateTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 (DEFAULT_TABLE table=default.Table_177f76165820429abc622c2444923017 [id=846642e389d8455184d0ce48466fcbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.057708 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb1fd17aa4a24b3a903c71482333ec84. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.058615 27123 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_11b878bc4a1740f2ab615579a62a5cfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.058713 27123 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11b878bc4a1740f2ab615579a62a5cfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.058848 27837 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.058974 27127 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_07a533b683494296ad4496688fa5fc47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.059057 27127 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_07a533b683494296ad4496688fa5fc47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.059185 27837 tablet_bootstrap.cc:654] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.059381 27126 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_7a4dd759006b4f999a36751c7b62ae48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.059465 27126 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a4dd759006b4f999a36751c7b62ae48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.059623 27839 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.059664 27839 ts_tablet_manager.cc:1367] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:08.059783 27839 raft_consensus.cc:348] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.059827 27839 raft_consensus.cc:374] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.059842 27839 raft_consensus.cc:729] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.059875 27839 consensus_queue.cc:260] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.059942 27839 ts_tablet_manager.cc:1398] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.060154 27701 tablet_service.cc:1467] Processing CreateTablet for tablet d59c77e3e1444b2fb211363d59bd0631 (DEFAULT_TABLE table=default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.060176 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 (DEFAULT_TABLE table=default.Table_c333427c445048efb408340ce1cdef66 [id=59ad58c5742d43efb6dc5161c7e8f41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.060233 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d59c77e3e1444b2fb211363d59bd0631. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.060348 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff3c14c40ac4d49a0609044ad300333. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.060609 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1e79e771df40f7abcdecaf683ae61b (DEFAULT_TABLE table=default.Table_9fc74e464bc14c979b42aa031fc368f4 [id=b2a8365635134977a758514aa7a4351d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.060688 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1e79e771df40f7abcdecaf683ae61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.061370 27429 tablet_service.cc:1467] Processing CreateTablet for tablet d59c77e3e1444b2fb211363d59bd0631 (DEFAULT_TABLE table=default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.061447 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d59c77e3e1444b2fb211363d59bd0631. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.061782 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1e79e771df40f7abcdecaf683ae61b (DEFAULT_TABLE table=default.Table_9fc74e464bc14c979b42aa031fc368f4 [id=b2a8365635134977a758514aa7a4351d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.061861 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1e79e771df40f7abcdecaf683ae61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.062233 27128 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c456a5e68e2f4baaaec502f69453d04b" } new_table_name: "default.Table_2caacb35bf3042b0a5e7074cd3e3f057" modify_external_catalogs: true
I20240503 02:09:08.062453 27839 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.062762 27839 tablet_bootstrap.cc:654] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.062754 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_177f76165820429abc622c2444923017" } new_table_name: "default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55" modify_external_catalogs: true
I20240503 02:09:08.062810 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 (DEFAULT_TABLE table=default.Table_c333427c445048efb408340ce1cdef66 [id=59ad58c5742d43efb6dc5161c7e8f41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.062880 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff3c14c40ac4d49a0609044ad300333. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.062906 27124 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_0bfcd325883a414484aec17b7af9cb2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.062979 27124 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bfcd325883a414484aec17b7af9cb2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.063709 27839 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.063751 27839 ts_tablet_manager.cc:1367] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.063884 27839 raft_consensus.cc:348] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.063946 27839 raft_consensus.cc:374] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.063967 27839 raft_consensus.cc:729] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.064000 27839 consensus_queue.cc:260] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.064095 27839 ts_tablet_manager.cc:1398] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.064159 27839 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.064417 27839 tablet_bootstrap.cc:654] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.064543 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 6b1e79e771df40f7abcdecaf683ae61b (DEFAULT_TABLE table=default.Table_9fc74e464bc14c979b42aa031fc368f4 [id=b2a8365635134977a758514aa7a4351d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.064581 27563 tablet_service.cc:1467] Processing CreateTablet for tablet d59c77e3e1444b2fb211363d59bd0631 (DEFAULT_TABLE table=default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.064641 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b1e79e771df40f7abcdecaf683ae61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.065021 27839 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.065063 27839 ts_tablet_manager.cc:1367] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.065178 27839 raft_consensus.cc:348] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.065225 27839 raft_consensus.cc:374] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.065244 27839 raft_consensus.cc:729] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.065269 27839 consensus_queue.cc:260] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.065313 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d59c77e3e1444b2fb211363d59bd0631. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.065393 27839 ts_tablet_manager.cc:1398] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.065445 27839 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:08.065539 27125 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb1fd17aa4a24b3a903c71482333ec84 consensus state has no leader
I20240503 02:09:08.065647 27566 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 (DEFAULT_TABLE table=default.Table_c333427c445048efb408340ce1cdef66 [id=59ad58c5742d43efb6dc5161c7e8f41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.065718 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff3c14c40ac4d49a0609044ad300333. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.065824 27839 tablet_bootstrap.cc:654] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.066820 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_2e984ea13c8c4a0d884abb238e6c64ae" } new_table_name: "default.Table_3d13fc887dfc4a33a3c4729168d769b3" modify_external_catalogs: true
I20240503 02:09:08.067077 27838 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.067121 27838 ts_tablet_manager.cc:1367] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240503 02:09:08.067242 27838 raft_consensus.cc:348] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.067296 27838 raft_consensus.cc:374] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.067314 27838 raft_consensus.cc:729] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.067348 27838 consensus_queue.cc:260] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.067425 27838 ts_tablet_manager.cc:1398] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.067466 27838 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.067795 27838 tablet_bootstrap.cc:654] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.068401 27839 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.068444 27839 ts_tablet_manager.cc:1367] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:08.068465 27917 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.068558 27839 raft_consensus.cc:348] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.068605 27839 raft_consensus.cc:374] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.068622 27839 raft_consensus.cc:729] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.068655 27839 consensus_queue.cc:260] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.068727 27839 ts_tablet_manager.cc:1398] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.068742 27893 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.068851 27838 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.068903 27838 ts_tablet_manager.cc:1367] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:08.069016 27838 raft_consensus.cc:348] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.069063 27838 raft_consensus.cc:374] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.069078 27838 raft_consensus.cc:729] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.069100 27838 consensus_queue.cc:260] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.069169 27838 ts_tablet_manager.cc:1398] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.069231 27838 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.069586 27838 tablet_bootstrap.cc:654] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.070088 27838 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.070132 27838 ts_tablet_manager.cc:1367] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:08.070266 27838 raft_consensus.cc:348] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.070312 27838 raft_consensus.cc:374] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.070330 27838 raft_consensus.cc:729] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.070375 27838 consensus_queue.cc:260] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.070468 27838 ts_tablet_manager.cc:1398] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.070523 27838 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.070837 27838 tablet_bootstrap.cc:654] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.071926 27838 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.071974 27838 ts_tablet_manager.cc:1367] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:08.072062 27127 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_9fc74e464bc14c979b42aa031fc368f4" } new_table_name: "default.Table_f54f404aa16e495d994c597b7ea98869" modify_external_catalogs: true
I20240503 02:09:08.072084 27838 raft_consensus.cc:348] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.072129 27838 raft_consensus.cc:374] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.072144 27838 raft_consensus.cc:729] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.072201 27892 tablet.cc:1722] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.072865 27837 tablet_bootstrap.cc:492] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.072914 27837 ts_tablet_manager.cc:1367] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240503 02:09:08.073029 27837 raft_consensus.cc:348] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.073084 27837 raft_consensus.cc:374] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.073102 27837 raft_consensus.cc:729] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.073136 27837 consensus_queue.cc:260] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.073213 27837 ts_tablet_manager.cc:1398] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.073266 27837 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.073621 27837 tablet_bootstrap.cc:654] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.073794 27429 tablet_service.cc:1467] Processing CreateTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea (DEFAULT_TABLE table=default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.073773 27838 consensus_queue.cc:260] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.073882 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc8839c6cb14c648d15ef51ccb722ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.074122 27838 ts_tablet_manager.cc:1398] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:08.074178 27838 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.074489 27838 tablet_bootstrap.cc:654] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.074510 27427 tablet_service.cc:1467] Processing CreateTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.074592 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3d3ca1b7ac478d8f5f942d6892d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.074813 27839 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.075131 27839 tablet_bootstrap.cc:654] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.075250 27703 tablet_service.cc:1467] Processing CreateTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.075330 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3d3ca1b7ac478d8f5f942d6892d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.075592 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 9c11ab6468184080a8697375a119547c (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.075663 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c11ab6468184080a8697375a119547c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.075752 27838 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.075788 27838 ts_tablet_manager.cc:1367] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:08.075904 27838 raft_consensus.cc:348] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.075950 27838 raft_consensus.cc:374] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.075966 27838 raft_consensus.cc:729] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.075999 27838 consensus_queue.cc:260] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.076082 27838 ts_tablet_manager.cc:1398] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.076335 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 9c11ab6468184080a8697375a119547c (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.076339 27839 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.076382 27839 ts_tablet_manager.cc:1367] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:08.076408 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c11ab6468184080a8697375a119547c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.076503 27839 raft_consensus.cc:348] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.076550 27839 raft_consensus.cc:374] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.076565 27839 raft_consensus.cc:729] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.076601 27839 consensus_queue.cc:260] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.076666 27839 ts_tablet_manager.cc:1398] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.076706 27839 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.077028 27839 tablet_bootstrap.cc:654] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.077316 27838 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.077383 27699 tablet_service.cc:1467] Processing CreateTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea (DEFAULT_TABLE table=default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.077450 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc8839c6cb14c648d15ef51ccb722ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.077626 27838 tablet_bootstrap.cc:654] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.078308 27838 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.078343 27838 ts_tablet_manager.cc:1367] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
W20240503 02:09:08.078449 27118 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_b286e68e663d42dfa92e9bb689b0fb4b [id=8a300650ad514265b389e710d1ee3446], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
I20240503 02:09:08.078478 27838 raft_consensus.cc:348] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.078526 27838 raft_consensus.cc:374] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.078541 27838 raft_consensus.cc:729] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.078577 27838 consensus_queue.cc:260] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.078595 27837 tablet_bootstrap.cc:492] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.078632 27837 ts_tablet_manager.cc:1367] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:08.078665 27838 ts_tablet_manager.cc:1398] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.078712 27838 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.078744 27837 raft_consensus.cc:348] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.078796 27837 raft_consensus.cc:374] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.078814 27837 raft_consensus.cc:729] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.078847 27837 consensus_queue.cc:260] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.078919 27837 ts_tablet_manager.cc:1398] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.078966 27837 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.079167 27838 tablet_bootstrap.cc:654] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.079295 27837 tablet_bootstrap.cc:654] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.079530 27125 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_b286e68e663d42dfa92e9bb689b0fb4b" } new_table_name: "default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1" modify_external_catalogs: true
I20240503 02:09:08.079638 27453 raft_consensus.cc:1261] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:08.079682 27123 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c333427c445048efb408340ce1cdef66" } modify_external_catalogs: true
I20240503 02:09:08.079723 27123 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_c333427c445048efb408340ce1cdef66" } modify_external_catalogs: true
I20240503 02:09:08.079773 27839 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.079806 27839 ts_tablet_manager.cc:1367] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:08.079924 27839 raft_consensus.cc:348] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.079973 27839 raft_consensus.cc:374] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.079946 27904 consensus_queue.cc:1035] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.079991 27839 raft_consensus.cc:729] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.080024 27839 consensus_queue.cc:260] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.080103 27839 ts_tablet_manager.cc:1398] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.080158 27839 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.080381 27122 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_440ae0e7eec44ba28808051dfb31ee04" } modify_external_catalogs: true
I20240503 02:09:08.080435 27122 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_440ae0e7eec44ba28808051dfb31ee04" } modify_external_catalogs: true
I20240503 02:09:08.080550 27839 tablet_bootstrap.cc:654] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.081104 27130 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_38ee0012453d4e54b8c173c8129d3b74" } modify_external_catalogs: true
I20240503 02:09:08.081169 27130 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_38ee0012453d4e54b8c173c8129d3b74" } modify_external_catalogs: true
I20240503 02:09:08.081576 27587 raft_consensus.cc:1261] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:08.081583 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 9c11ab6468184080a8697375a119547c (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.081676 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c11ab6468184080a8697375a119547c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.081707 27564 tablet_service.cc:1467] Processing CreateTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea (DEFAULT_TABLE table=default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.081772 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc8839c6cb14c648d15ef51ccb722ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.081746 27904 consensus_queue.cc:1035] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.081852 27126 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_2caacb35bf3042b0a5e7074cd3e3f057 [id=453d4c39809a42f38267db3c4d45748a] alter complete (version 2)
W20240503 02:09:08.081943 27127 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b1e79e771df40f7abcdecaf683ae61b consensus state has no leader
I20240503 02:09:08.081677 27566 tablet_service.cc:1467] Processing CreateTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.082726 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd3d3ca1b7ac478d8f5f942d6892d600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.083060 27837 tablet_bootstrap.cc:492] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.083097 27837 ts_tablet_manager.cc:1367] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
W20240503 02:09:08.083202 27125 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
I20240503 02:09:08.083210 27837 raft_consensus.cc:348] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.083259 27837 raft_consensus.cc:374] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.083276 27837 raft_consensus.cc:729] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.083276 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 16577f3067c04965887c8b3db12795e4
I20240503 02:09:08.083323 27837 consensus_queue.cc:260] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.083427 27837 ts_tablet_manager.cc:1398] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.083492 27837 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.083825 27128 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.083796 27837 tablet_bootstrap.cc:654] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.084064 27127 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.084834 27838 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.084877 27838 ts_tablet_manager.cc:1367] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:08.084993 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 5443787e1d464baf93549b49571f5687 (DEFAULT_TABLE table=default.Table_0bfcd325883a414484aec17b7af9cb2f [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.085000 27838 raft_consensus.cc:348] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.085050 27838 raft_consensus.cc:374] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.085069 27838 raft_consensus.cc:729] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.085083 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5443787e1d464baf93549b49571f5687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.085103 27838 consensus_queue.cc:260] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.085192 27838 ts_tablet_manager.cc:1398] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.085250 27838 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.085486 27127 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4653432f1d9041cc83d5f0beb9ac8b6d" } modify_external_catalogs: true
I20240503 02:09:08.085538 27127 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_4653432f1d9041cc83d5f0beb9ac8b6d" } modify_external_catalogs: true
I20240503 02:09:08.085601 27838 tablet_bootstrap.cc:654] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.086272 27125 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41820:
name: "default.Table_dc8099703c974c61a042d8d2c291a023"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:08.086365 27125 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc8099703c974c61a042d8d2c291a023 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:08.086462 27839 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.086499 27839 ts_tablet_manager.cc:1367] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:08.086619 27839 raft_consensus.cc:348] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.086670 27839 raft_consensus.cc:374] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.086687 27839 raft_consensus.cc:729] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.086719 27839 consensus_queue.cc:260] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.086804 27839 ts_tablet_manager.cc:1398] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.086858 27839 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.087149 27839 tablet_bootstrap.cc:654] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.087213 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 5443787e1d464baf93549b49571f5687 (DEFAULT_TABLE table=default.Table_0bfcd325883a414484aec17b7af9cb2f [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.087266 27130 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 960d68053f9642ff82c9c30983973e93
I20240503 02:09:08.087294 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5443787e1d464baf93549b49571f5687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.087927 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 5443787e1d464baf93549b49571f5687 (DEFAULT_TABLE table=default.Table_0bfcd325883a414484aec17b7af9cb2f [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.088013 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5443787e1d464baf93549b49571f5687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.088163 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 960d68053f9642ff82c9c30983973e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.088230 27845 tablet_replica.cc:330] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.088272 27845 raft_consensus.cc:2229] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.088311 27845 raft_consensus.cc:2258] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.088606 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 16577f3067c04965887c8b3db12795e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:08.088641 27845 ts_tablet_manager.cc:1875] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.088688 27844 tablet_replica.cc:330] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:08.088744 27844 raft_consensus.cc:2229] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.088753 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 960d68053f9642ff82c9c30983973e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:08.088774 27844 raft_consensus.cc:2258] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.089079 27844 ts_tablet_manager.cc:1875] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.089800 27845 ts_tablet_manager.cc:1888] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:08.089854 27845 log.cc:1196] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/960d68053f9642ff82c9c30983973e93
I20240503 02:09:08.090018 27845 ts_tablet_manager.cc:1909] T 960d68053f9642ff82c9c30983973e93 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.090332 27844 ts_tablet_manager.cc:1888] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.090378 27844 log.cc:1196] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/16577f3067c04965887c8b3db12795e4
I20240503 02:09:08.090518 27844 ts_tablet_manager.cc:1909] T 16577f3067c04965887c8b3db12795e4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:08.090684 27839 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.090724 27839 ts_tablet_manager.cc:1367] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:08.090853 27839 raft_consensus.cc:348] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.090912 27839 raft_consensus.cc:374] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.090930 27839 raft_consensus.cc:729] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.090942 27893 tablet.cc:1722] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:08.090965 27839 consensus_queue.cc:260] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.091054 27839 ts_tablet_manager.cc:1398] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.091543 27838 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.091580 27838 ts_tablet_manager.cc:1367] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:08.091691 27838 raft_consensus.cc:348] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.091743 27838 raft_consensus.cc:374] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.091759 27838 raft_consensus.cc:729] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.091781 27838 consensus_queue.cc:260] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.091854 27838 ts_tablet_manager.cc:1398] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.091899 27838 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.092168 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 16577f3067c04965887c8b3db12795e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.092231 27838 tablet_bootstrap.cc:654] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.092248 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 16577f3067c04965887c8b3db12795e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.092305 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 960d68053f9642ff82c9c30983973e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.092312 27843 tablet_replica.cc:330] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.092381 27843 raft_consensus.cc:2229] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.092409 27843 raft_consensus.cc:2258] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.092475 27845 tablet_replica.cc:330] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.092523 27845 raft_consensus.cc:2229] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.092561 27845 pending_rounds.cc:62] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:08.092577 27845 pending_rounds.cc:68] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419998141739008 op_type: NO_OP noop_request { }
I20240503 02:09:08.092608 27845 raft_consensus.cc:2866] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:08.092595 27843 ts_tablet_manager.cc:1875] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.092628 27845 raft_consensus.cc:2258] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.092793 27846 raft_consensus.cc:482] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.092828 27846 raft_consensus.cc:504] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.092919 27846 leader_election.cc:288] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.092976 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]) successfully deleted
I20240503 02:09:08.093204 27837 tablet_bootstrap.cc:492] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.093238 27837 ts_tablet_manager.cc:1367] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:08.093400 27837 raft_consensus.cc:348] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.093451 27837 raft_consensus.cc:374] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.093468 27837 raft_consensus.cc:729] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.093475 27844 tablet_replica.cc:330] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:08.093493 27837 consensus_queue.cc:260] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.093516 27844 raft_consensus.cc:2229] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.093560 27844 raft_consensus.cc:2258] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.093560 27837 ts_tablet_manager.cc:1398] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.093735 27837 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.093808 27843 ts_tablet_manager.cc:1888] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.093853 27843 log.cc:1196] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/16577f3067c04965887c8b3db12795e4
I20240503 02:09:08.093922 27838 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.093958 27838 ts_tablet_manager.cc:1367] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:08.093989 27843 ts_tablet_manager.cc:1909] T 16577f3067c04965887c8b3db12795e4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.094071 27837 tablet_bootstrap.cc:654] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.094070 27838 raft_consensus.cc:348] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.094123 27838 raft_consensus.cc:374] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.094142 27838 raft_consensus.cc:729] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.094179 27838 consensus_queue.cc:260] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.094249 27838 ts_tablet_manager.cc:1398] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.094331 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.094374 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]). Expected version 1 got 0
I20240503 02:09:08.094431 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.094743 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1fd17aa4a24b3a903c71482333ec84" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.094802 27588 raft_consensus.cc:2454] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.094833 27837 tablet_bootstrap.cc:492] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.094871 27837 ts_tablet_manager.cc:1367] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.094933 27128 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]). Expected version 1 got 0
I20240503 02:09:08.095005 27837 raft_consensus.cc:348] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.095057 27837 raft_consensus.cc:374] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.095072 27837 raft_consensus.cc:729] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.095075 27845 ts_tablet_manager.cc:1875] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.095105 27837 consensus_queue.cc:260] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.095129 27386 leader_election.cc:302] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.095189 27837 ts_tablet_manager.cc:1398] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.095206 27846 raft_consensus.cc:2781] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.095230 27846 raft_consensus.cc:482] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.095232 27837 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.095248 27846 raft_consensus.cc:3037] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.095528 27837 tablet_bootstrap.cc:654] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.095824 27846 raft_consensus.cc:504] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.095927 27846 leader_election.cc:288] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.096028 27115 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 16577f3067c04965887c8b3db12795e4 (table default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]) successfully deleted
I20240503 02:09:08.096194 27845 ts_tablet_manager.cc:1888] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:08.096244 27845 log.cc:1196] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/16577f3067c04965887c8b3db12795e4
I20240503 02:09:08.096321 27892 tablet.cc:1722] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:08.096414 27845 ts_tablet_manager.cc:1909] T 16577f3067c04965887c8b3db12795e4 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.096427 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.096519 27123 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 7ff3c14c40ac4d49a0609044ad300333
I20240503 02:09:08.096932 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 16577f3067c04965887c8b3db12795e4 (table default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]) successfully deleted
I20240503 02:09:08.097314 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.097400 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.097442 27900 raft_consensus.cc:482] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.097476 27900 raft_consensus.cc:504] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.097507 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1fd17aa4a24b3a903c71482333ec84" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.097573 27722 raft_consensus.cc:2454] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.097612 27844 ts_tablet_manager.cc:1875] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.097638 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1fd17aa4a24b3a903c71482333ec84" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.097676 27723 raft_consensus.cc:3037] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.098249 27723 raft_consensus.cc:2454] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.097576 27900 leader_election.cc:288] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.098429 27845 tablet_replica.cc:330] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.098474 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7a66cc87834e739e958a2caf73de0c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.098534 27453 raft_consensus.cc:2454] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.098582 27844 ts_tablet_manager.cc:1888] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:08.098616 27844 log.cc:1196] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/960d68053f9642ff82c9c30983973e93
I20240503 02:09:08.098794 27844 ts_tablet_manager.cc:1909] T 960d68053f9642ff82c9c30983973e93 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:08.099069 27127 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 630a06d78f984c27903d401de9e7247a
I20240503 02:09:08.099247 27659 leader_election.cc:302] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.099321 27900 raft_consensus.cc:2781] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.099346 27900 raft_consensus.cc:482] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.099362 27900 raft_consensus.cc:3037] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.099545 27388 leader_election.cc:302] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.099658 28054 raft_consensus.cc:2781] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.099704 28054 raft_consensus.cc:686] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.099738 28054 consensus_queue.cc:237] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.099896 27900 raft_consensus.cc:504] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.099993 27900 leader_election.cc:288] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.100380 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7a66cc87834e739e958a2caf73de0c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.100443 27453 raft_consensus.cc:3037] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.100864 27453 raft_consensus.cc:2454] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.100997 27659 leader_election.cc:302] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.101004 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 630a06d78f984c27903d401de9e7247a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.101079 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 630a06d78f984c27903d401de9e7247a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.101203 27900 raft_consensus.cc:2781] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.101261 27900 raft_consensus.cc:686] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.101301 27900 consensus_queue.cc:237] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.098500 27845 raft_consensus.cc:2229] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.101820 27845 raft_consensus.cc:2258] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.102226 27845 ts_tablet_manager.cc:1875] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.103294 27127 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d] alter complete (version 1)
I20240503 02:09:08.103320 27900 raft_consensus.cc:482] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.103363 27900 raft_consensus.cc:504] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.103453 27900 leader_election.cc:288] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.103588 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.103638 27453 raft_consensus.cc:2454] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.103744 27659 leader_election.cc:302] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.103803 27900 raft_consensus.cc:2781] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.103830 27900 raft_consensus.cc:482] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.103847 27900 raft_consensus.cc:3037] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.103891 27845 ts_tablet_manager.cc:1888] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.103925 27845 log.cc:1196] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/7ff3c14c40ac4d49a0609044ad300333
I20240503 02:09:08.104120 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (DEFAULT_TABLE table=default.Table_dc8099703c974c61a042d8d2c291a023 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.104216 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1fbd4e5618451fac90622fbcf1e9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.104236 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (DEFAULT_TABLE table=default.Table_dc8099703c974c61a042d8d2c291a023 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.104343 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1fbd4e5618451fac90622fbcf1e9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.104581 27900 raft_consensus.cc:504] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.104676 27900 leader_election.cc:288] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.105217 27839 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.105576 27839 tablet_bootstrap.cc:654] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.105604 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.105652 27453 raft_consensus.cc:3037] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.105815 27124 catalog_manager.cc:5484] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.105880 27838 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.105913 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.106187 27838 tablet_bootstrap.cc:654] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.106209 27453 raft_consensus.cc:2454] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.106338 27659 leader_election.cc:302] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.106312 27127 catalog_manager.cc:5484] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.106604 27900 raft_consensus.cc:2781] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.106650 27900 raft_consensus.cc:686] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.106683 27900 consensus_queue.cc:237] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.107332 27845 ts_tablet_manager.cc:1909] T 7ff3c14c40ac4d49a0609044ad300333 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.107637 27845 tablet_replica.cc:330] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.107734 27845 raft_consensus.cc:2229] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.107911 27845 raft_consensus.cc:2258] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.107944 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7a66cc87834e739e958a2caf73de0c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.108001 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7a66cc87834e739e958a2caf73de0c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.107964 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.108017 27585 raft_consensus.cc:3037] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.108121 27845 ts_tablet_manager.cc:1875] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.108106 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.108198 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (DEFAULT_TABLE table=default.Table_dc8099703c974c61a042d8d2c291a023 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.108275 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1fbd4e5618451fac90622fbcf1e9bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.108338 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff3c14c40ac4d49a0609044ad300333 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:08.108417 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet 630a06d78f984c27903d401de9e7247a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:42354
I20240503 02:09:08.108608 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 7ff3c14c40ac4d49a0609044ad300333 (table default.Table_c333427c445048efb408340ce1cdef66 [id=59ad58c5742d43efb6dc5161c7e8f41e]) successfully deleted
I20240503 02:09:08.108709 27585 raft_consensus.cc:2454] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
W20240503 02:09:08.108839 27656 leader_election.cc:341] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:09:08.108951 27656 leader_election.cc:341] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:08.109253 27127 catalog_manager.cc:5484] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.109400 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb1fd17aa4a24b3a903c71482333ec84" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.109449 27582 raft_consensus.cc:3037] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.109458 27845 ts_tablet_manager.cc:1888] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.109530 27845 log.cc:1196] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/630a06d78f984c27903d401de9e7247a
I20240503 02:09:08.109735 27845 ts_tablet_manager.cc:1909] T 630a06d78f984c27903d401de9e7247a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.109972 27843 tablet_replica.cc:330] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.110005 27582 raft_consensus.cc:2454] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.110034 27843 raft_consensus.cc:2229] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.110072 27843 raft_consensus.cc:2258] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.110129 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 630a06d78f984c27903d401de9e7247a (table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]) successfully deleted
I20240503 02:09:08.110258 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 16577f3067c04965887c8b3db12795e4 (table default.Table_440ae0e7eec44ba28808051dfb31ee04 [id=05b29fa446104f92b216de3cfa8116c0]) successfully deleted
I20240503 02:09:08.110689 27843 ts_tablet_manager.cc:1875] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.110787 27837 tablet_bootstrap.cc:492] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.110827 27837 ts_tablet_manager.cc:1367] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:08.110987 27837 raft_consensus.cc:348] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.111044 27837 raft_consensus.cc:374] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.111063 27837 raft_consensus.cc:729] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.111104 27837 consensus_queue.cc:260] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.111191 27837 ts_tablet_manager.cc:1398] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.111253 27837 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.111583 27837 tablet_bootstrap.cc:654] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.111900 27843 ts_tablet_manager.cc:1888] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:08.111941 27843 log.cc:1196] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/960d68053f9642ff82c9c30983973e93
I20240503 02:09:08.112123 27843 ts_tablet_manager.cc:1909] T 960d68053f9642ff82c9c30983973e93 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.112707 27124 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_2caacb35bf3042b0a5e7074cd3e3f057" } modify_external_catalogs: true
I20240503 02:09:08.112773 27124 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41820:
table { table_name: "default.Table_2caacb35bf3042b0a5e7074cd3e3f057" } modify_external_catalogs: true
I20240503 02:09:08.112969 27843 tablet_replica.cc:330] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.113029 27843 raft_consensus.cc:2229] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.113057 27843 raft_consensus.cc:2258] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.113052 27837 tablet_bootstrap.cc:492] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.113093 27837 ts_tablet_manager.cc:1367] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:08.113209 27843 ts_tablet_manager.cc:1875] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.113214 27837 raft_consensus.cc:348] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.113265 27837 raft_consensus.cc:374] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.113282 27837 raft_consensus.cc:729] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.113337 27837 consensus_queue.cc:260] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.113394 27844 tablet_replica.cc:330] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:08.113416 27837 ts_tablet_manager.cc:1398] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.113444 27844 raft_consensus.cc:2229] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.113471 27844 raft_consensus.cc:2258] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.113476 27837 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.113673 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 960d68053f9642ff82c9c30983973e93 (table default.Table_38ee0012453d4e54b8c173c8129d3b74 [id=2a045f55a9c84ca88188f62b60a8cb2d]) successfully deleted
I20240503 02:09:08.114128 27837 tablet_bootstrap.cc:654] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.114892 27844 ts_tablet_manager.cc:1875] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.115239 27843 ts_tablet_manager.cc:1888] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.115243 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:08.115443 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 85e1f031e13f47c692de9a8a5e256613 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.115558 27845 tablet_replica.cc:330] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.115700 27845 raft_consensus.cc:2229] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.115778 27845 raft_consensus.cc:2258] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.115959 27837 tablet_bootstrap.cc:492] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.116014 27837 ts_tablet_manager.cc:1367] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:08.116098 27844 ts_tablet_manager.cc:1888] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.116147 27844 log.cc:1196] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/7ff3c14c40ac4d49a0609044ad300333
I20240503 02:09:08.116271 27845 ts_tablet_manager.cc:1875] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.116305 27844 ts_tablet_manager.cc:1909] T 7ff3c14c40ac4d49a0609044ad300333 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:08.116127 27837 raft_consensus.cc:348] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.116709 27837 raft_consensus.cc:374] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.116742 27837 raft_consensus.cc:729] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.116775 27837 consensus_queue.cc:260] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.116856 27837 ts_tablet_manager.cc:1398] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:08.116923 27837 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.117937 27837 tablet_bootstrap.cc:654] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.118105 27845 ts_tablet_manager.cc:1888] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:08.118374 27845 log.cc:1196] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:08.118644 27845 ts_tablet_manager.cc:1909] T 85e1f031e13f47c692de9a8a5e256613 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.118789 27837 tablet_bootstrap.cc:492] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.118834 27837 ts_tablet_manager.cc:1367] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:08.118959 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_2caacb35bf3042b0a5e7074cd3e3f057 [id=453d4c39809a42f38267db3c4d45748a]) successfully deleted
I20240503 02:09:08.119042 27844 tablet_replica.cc:330] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:08.119107 27844 raft_consensus.cc:2229] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.119159 27844 pending_rounds.cc:62] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:08.119227 27844 pending_rounds.cc:68] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419997808558080 op_type: NO_OP noop_request { }
I20240503 02:09:08.118958 27837 raft_consensus.cc:348] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.119318 27844 raft_consensus.cc:2866] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:08.119339 27837 raft_consensus.cc:374] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.119659 27837 raft_consensus.cc:729] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.119711 27122 master_service.cc:946] ReplaceTablet: received request to replace tablet 203665395f3745f196fb393dd82b8fc9 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:08.119625 27844 raft_consensus.cc:2258] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.119727 27837 consensus_queue.cc:260] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.120199 27837 ts_tablet_manager.cc:1398] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.120260 27837 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.120391 27844 ts_tablet_manager.cc:1875] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.120934 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:08.121001 27122 catalog_manager.cc:6554] ReplaceTablet: tablet 203665395f3745f196fb393dd82b8fc9 deleted and replaced by tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:08.121351 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 203665395f3745f196fb393dd82b8fc9 with delete_type TABLET_DATA_DELETED (replaced by tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.121701 27837 tablet_bootstrap.cc:654] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.122347 27844 ts_tablet_manager.cc:1888] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:08.122444 27844 log.cc:1196] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/630a06d78f984c27903d401de9e7247a
I20240503 02:09:08.122678 27844 ts_tablet_manager.cc:1909] T 630a06d78f984c27903d401de9e7247a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:08.122712 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (DEFAULT_TABLE table=default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.122797 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.115613 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 85e1f031e13f47c692de9a8a5e256613 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:08 UTC) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.122977 27843 log.cc:1196] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/7ff3c14c40ac4d49a0609044ad300333
I20240503 02:09:08.121227 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet 203665395f3745f196fb393dd82b8fc9 with delete_type TABLET_DATA_DELETED (replaced by tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.122648 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (DEFAULT_TABLE table=default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.123085 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.123142 27843 ts_tablet_manager.cc:1909] T 7ff3c14c40ac4d49a0609044ad300333 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.123325 27843 tablet_replica.cc:330] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.123374 27843 raft_consensus.cc:2229] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.123402 27843 raft_consensus.cc:2258] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.123544 27845 tablet_replica.cc:330] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.123592 27845 raft_consensus.cc:2229] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.123636 27845 pending_rounds.cc:62] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:08.123653 27845 pending_rounds.cc:68] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419998142205952 op_type: NO_OP noop_request { }
I20240503 02:09:08.123687 27845 raft_consensus.cc:2866] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:08.123708 27845 raft_consensus.cc:2258] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.123826 27843 ts_tablet_manager.cc:1875] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.124110 27837 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:08.124151 27837 ts_tablet_manager.cc:1367] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:08.124186 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 7ff3c14c40ac4d49a0609044ad300333 (table default.Table_c333427c445048efb408340ce1cdef66 [id=59ad58c5742d43efb6dc5161c7e8f41e]) successfully deleted
I20240503 02:09:08.124266 27837 raft_consensus.cc:348] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.124313 27837 raft_consensus.cc:374] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.124328 27837 raft_consensus.cc:729] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.124361 27837 consensus_queue.cc:260] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.124436 27837 ts_tablet_manager.cc:1398] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.125092 27845 ts_tablet_manager.cc:1875] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.126555 27845 ts_tablet_manager.cc:1888] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:08.126727 27122 master_service.cc:946] ReplaceTablet: received request to replace tablet 8127ac16597e40fa9b1590a61430aea0 from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:08.127175 27843 ts_tablet_manager.cc:1888] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.127189 27124 master_service.cc:946] ReplaceTablet: received request to replace tablet be7a66cc87834e739e958a2caf73de0c from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:08.127220 27843 log.cc:1196] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/630a06d78f984c27903d401de9e7247a
I20240503 02:09:08.127360 27843 ts_tablet_manager.cc:1909] T 630a06d78f984c27903d401de9e7247a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.127447 27845 log.cc:1196] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/203665395f3745f196fb393dd82b8fc9
I20240503 02:09:08.127730 27843 tablet_replica.cc:330] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.127804 27122 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:08.127876 27122 catalog_manager.cc:6554] ReplaceTablet: tablet 8127ac16597e40fa9b1590a61430aea0 deleted and replaced by tablet 092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:08.128000 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 8127ac16597e40fa9b1590a61430aea0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092ad3ae09fe47bbbe473643060c43d0) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.128036 27845 ts_tablet_manager.cc:1909] T 203665395f3745f196fb393dd82b8fc9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.128283 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 630a06d78f984c27903d401de9e7247a (table default.Table_4653432f1d9041cc83d5f0beb9ac8b6d [id=4537c4763bb746f2b4ab05ef06b26425]) successfully deleted
I20240503 02:09:08.128289 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 203665395f3745f196fb393dd82b8fc9 (table default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]) successfully deleted
I20240503 02:09:08.128386 27843 raft_consensus.cc:2229] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.128443 27843 raft_consensus.cc:2258] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.128473 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 8127ac16597e40fa9b1590a61430aea0 with delete_type TABLET_DATA_DELETED (replaced by tablet 092ad3ae09fe47bbbe473643060c43d0) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.128510 27845 tablet_replica.cc:330] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.128563 27845 raft_consensus.cc:2229] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.128600 27845 raft_consensus.cc:2258] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.128782 27845 ts_tablet_manager.cc:1875] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.128937 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:08.129024 27124 catalog_manager.cc:6554] ReplaceTablet: tablet be7a66cc87834e739e958a2caf73de0c deleted and replaced by tablet 3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:08.129112 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet be7a66cc87834e739e958a2caf73de0c with delete_type TABLET_DATA_DELETED (replaced by tablet 3a12a765f85d43f180a9ce0254c2fd10) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.129457 27843 ts_tablet_manager.cc:1875] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.129793 27124 master_service.cc:946] ReplaceTablet: received request to replace tablet 38d89374090345758db303c33b460c9a from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:08.129873 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet be7a66cc87834e739e958a2caf73de0c with delete_type TABLET_DATA_DELETED (replaced by tablet 3a12a765f85d43f180a9ce0254c2fd10) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.130172 27845 ts_tablet_manager.cc:1888] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.130276 27845 log.cc:1196] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:08.130476 27845 ts_tablet_manager.cc:1909] T 8127ac16597e40fa9b1590a61430aea0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.130786 27845 tablet_replica.cc:330] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.130894 27845 raft_consensus.cc:2229] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.130982 27845 raft_consensus.cc:2258] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.131103 27843 ts_tablet_manager.cc:1888] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:08.131203 27843 log.cc:1196] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:08.131193 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:08.131244 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 092ad3ae09fe47bbbe473643060c43d0 (DEFAULT_TABLE table=default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.131328 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092ad3ae09fe47bbbe473643060c43d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.131556 27124 catalog_manager.cc:5771] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Sending DeleteTablet for 3 replicas of tablet 38d89374090345758db303c33b460c9a
I20240503 02:09:08.131605 27124 catalog_manager.cc:6554] ReplaceTablet: tablet 38d89374090345758db303c33b460c9a deleted and replaced by tablet f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:08.131443 27843 ts_tablet_manager.cc:1909] T 85e1f031e13f47c692de9a8a5e256613 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.132026 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 85e1f031e13f47c692de9a8a5e256613 (table default.Table_2caacb35bf3042b0a5e7074cd3e3f057 [id=453d4c39809a42f38267db3c4d45748a]) successfully deleted
I20240503 02:09:08.132067 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 38d89374090345758db303c33b460c9a with delete_type TABLET_DATA_DELETED (replaced by tablet f4f54b533856464e81d77e726b2b7f31) from {username='slave'} at 127.0.0.1:43166
I20240503 02:09:08.132107 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 3a12a765f85d43f180a9ce0254c2fd10 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.132107 27843 tablet_replica.cc:330] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.132191 27843 raft_consensus.cc:2229] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.132208 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12a765f85d43f180a9ce0254c2fd10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.132222 27843 raft_consensus.cc:2258] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.132400 27843 ts_tablet_manager.cc:1875] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.132526 27845 ts_tablet_manager.cc:1875] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.133474 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 092ad3ae09fe47bbbe473643060c43d0 (DEFAULT_TABLE table=default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.133579 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092ad3ae09fe47bbbe473643060c43d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.133800 27845 ts_tablet_manager.cc:1888] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.133899 27845 log.cc:1196] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:08.134143 27845 ts_tablet_manager.cc:1909] T be7a66cc87834e739e958a2caf73de0c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.134299 27843 ts_tablet_manager.cc:1888] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.134380 27843 log.cc:1196] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/203665395f3745f196fb393dd82b8fc9
I20240503 02:09:08.134563 27843 ts_tablet_manager.cc:1909] T 203665395f3745f196fb393dd82b8fc9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.134582 27699 tablet_service.cc:1467] Processing CreateTablet for tablet f4f54b533856464e81d77e726b2b7f31 (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.134665 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f54b533856464e81d77e726b2b7f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.134773 27843 tablet_replica.cc:330] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.134809 27843 raft_consensus.cc:2229] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.134832 27843 raft_consensus.cc:2258] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.134963 27839 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.135003 27839 ts_tablet_manager.cc:1367] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240503 02:09:08.135007 27843 ts_tablet_manager.cc:1875] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.135149 27839 raft_consensus.cc:348] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.135205 27839 raft_consensus.cc:374] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.135224 27839 raft_consensus.cc:729] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.135262 27839 consensus_queue.cc:260] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.135336 27839 ts_tablet_manager.cc:1398] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.135407 27839 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.135816 27839 tablet_bootstrap.cc:654] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.136018 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 203665395f3745f196fb393dd82b8fc9 (table default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]) successfully deleted
I20240503 02:09:08.136205 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet be7a66cc87834e739e958a2caf73de0c (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:08.136379 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 38d89374090345758db303c33b460c9a with delete_type TABLET_DATA_DELETED (replaced by tablet f4f54b533856464e81d77e726b2b7f31) from {username='slave'} at 127.0.0.1:34410
I20240503 02:09:08.136504 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 3a12a765f85d43f180a9ce0254c2fd10 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.136587 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12a765f85d43f180a9ce0254c2fd10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.137074 27124 master_service.cc:946] ReplaceTablet: received request to replace tablet 38d89374090345758db303c33b460c9a from {username='slave'} at 127.0.0.1:41820
I20240503 02:09:08.137462 27429 tablet_service.cc:1467] Processing CreateTablet for tablet f4f54b533856464e81d77e726b2b7f31 (DEFAULT_TABLE table=default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:08.137547 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f54b533856464e81d77e726b2b7f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:08.137575 27843 ts_tablet_manager.cc:1888] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.137687 27843 log.cc:1196] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:08.137830 27843 ts_tablet_manager.cc:1909] T 8127ac16597e40fa9b1590a61430aea0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.137872 27839 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.137907 27839 ts_tablet_manager.cc:1367] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:08.138042 27839 raft_consensus.cc:348] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.138092 27839 raft_consensus.cc:374] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.138111 27839 raft_consensus.cc:729] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.138139 27839 consensus_queue.cc:260] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.138180 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:08.138216 27839 ts_tablet_manager.cc:1398] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.138273 27839 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.138247 27843 tablet_replica.cc:330] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.138500 27843 raft_consensus.cc:2229] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.138545 27843 pending_rounds.cc:62] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:08.138573 27839 tablet_bootstrap.cc:654] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.138561 27843 pending_rounds.cc:68] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419998623277056 op_type: NO_OP noop_request { }
I20240503 02:09:08.138598 27843 raft_consensus.cc:2866] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:08.138617 27843 raft_consensus.cc:2258] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.139240 27843 ts_tablet_manager.cc:1875] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.139757 27845 tablet_replica.cc:330] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:08.139854 27845 raft_consensus.cc:2229] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:08.139923 27845 pending_rounds.cc:62] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:08.139966 27845 pending_rounds.cc:68] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023419997577748480 op_type: NO_OP noop_request { }
I20240503 02:09:08.140010 27845 raft_consensus.cc:2866] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:08.140031 27845 raft_consensus.cc:2258] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.140277 27845 ts_tablet_manager.cc:1875] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.140518 27843 ts_tablet_manager.cc:1888] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:08.140573 27843 log.cc:1196] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:08.140743 27843 ts_tablet_manager.cc:1909] T be7a66cc87834e739e958a2caf73de0c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.141003 27843 tablet_replica.cc:330] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:08.141140 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet be7a66cc87834e739e958a2caf73de0c (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:08.141258 27843 raft_consensus.cc:2229] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:08.141351 27843 raft_consensus.cc:2258] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:08.141410 27917 tablet.cc:1722] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:08.141881 27843 ts_tablet_manager.cc:1875] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:08.142680 27839 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.142724 27839 ts_tablet_manager.cc:1367] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:08.142835 27839 raft_consensus.cc:348] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.142904 27839 raft_consensus.cc:374] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.142922 27839 raft_consensus.cc:729] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.142953 27839 consensus_queue.cc:260] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.143057 27839 ts_tablet_manager.cc:1398] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.143134 27839 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.143448 27839 tablet_bootstrap.cc:654] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.143728 27843 ts_tablet_manager.cc:1888] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:08.143767 27843 log.cc:1196] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/38d89374090345758db303c33b460c9a
I20240503 02:09:08.143899 27843 ts_tablet_manager.cc:1909] T 38d89374090345758db303c33b460c9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:08.144383 27845 ts_tablet_manager.cc:1888] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:08.144426 27845 log.cc:1196] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/38d89374090345758db303c33b460c9a
I20240503 02:09:08.144433 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
W20240503 02:09:08.144515 27122 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 092ad3ae09fe47bbbe473643060c43d0 consensus state has no leader
I20240503 02:09:08.144596 27845 ts_tablet_manager.cc:1909] T 38d89374090345758db303c33b460c9a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:08.144927 27117 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 38d89374090345758db303c33b460c9a (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:08.145164 27118 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 38d89374090345758db303c33b460c9a (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:08.145289 27839 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.145344 27839 ts_tablet_manager.cc:1367] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:08.145463 27839 raft_consensus.cc:348] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.145490 27838 tablet_bootstrap.cc:492] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.145510 27839 raft_consensus.cc:374] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.145529 27839 raft_consensus.cc:729] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.145530 27838 ts_tablet_manager.cc:1367] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20240503 02:09:08.145563 27839 consensus_queue.cc:260] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.145637 27839 ts_tablet_manager.cc:1398] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.145696 27839 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:08.145663 27838 raft_consensus.cc:348] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.145725 27838 raft_consensus.cc:374] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.145745 27838 raft_consensus.cc:729] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.145785 27838 consensus_queue.cc:260] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.145879 27838 ts_tablet_manager.cc:1398] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.145941 27838 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.146019 27839 tablet_bootstrap.cc:654] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.146275 27838 tablet_bootstrap.cc:654] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.146569 27839 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:08.146610 27839 ts_tablet_manager.cc:1367] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:08.146718 27839 raft_consensus.cc:348] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.146780 27839 raft_consensus.cc:374] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.146797 27839 raft_consensus.cc:729] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.146829 27839 consensus_queue.cc:260] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.146907 27839 ts_tablet_manager.cc:1398] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.147145 27838 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.147184 27838 ts_tablet_manager.cc:1367] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.147305 27838 raft_consensus.cc:348] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.147353 27838 raft_consensus.cc:374] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.147369 27838 raft_consensus.cc:729] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.147401 27838 consensus_queue.cc:260] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.147470 27838 ts_tablet_manager.cc:1398] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.147540 27838 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.147874 27838 tablet_bootstrap.cc:654] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.148365 27838 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.148404 27838 ts_tablet_manager.cc:1367] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:08.148535 27838 raft_consensus.cc:348] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.148588 27838 raft_consensus.cc:374] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.148605 27838 raft_consensus.cc:729] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.148643 27838 consensus_queue.cc:260] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.148720 27838 ts_tablet_manager.cc:1398] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.148768 27838 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.149106 27838 tablet_bootstrap.cc:654] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.150014 27838 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.150058 27838 ts_tablet_manager.cc:1367] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:08.150322 27838 raft_consensus.cc:348] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.150388 27838 raft_consensus.cc:374] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.150413 27838 raft_consensus.cc:729] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.150450 27838 consensus_queue.cc:260] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.150559 27838 ts_tablet_manager.cc:1398] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.150627 27838 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:08.151072 27838 tablet_bootstrap.cc:654] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:08.151697 27838 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:08.151741 27838 ts_tablet_manager.cc:1367] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:08.151852 27838 raft_consensus.cc:348] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.151908 27838 raft_consensus.cc:374] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:08.151924 27838 raft_consensus.cc:729] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.151968 27838 consensus_queue.cc:260] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.152042 27838 ts_tablet_manager.cc:1398] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:08.152333 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:08.152870 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:08.157284 27846 raft_consensus.cc:482] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.157363 27846 raft_consensus.cc:504] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.157470 27846 leader_election.cc:288] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.157615 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092ad3ae09fe47bbbe473643060c43d0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.157675 27723 raft_consensus.cc:2454] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:08.157797 27386 leader_election.cc:341] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:08.157843 27386 leader_election.cc:302] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.157920 27846 raft_consensus.cc:2781] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.157943 27846 raft_consensus.cc:482] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.157953 27846 raft_consensus.cc:3037] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.158353 27846 raft_consensus.cc:504] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.158442 27846 leader_election.cc:288] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.158612 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092ad3ae09fe47bbbe473643060c43d0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.158668 27723 raft_consensus.cc:3037] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:08.158787 27386 leader_election.cc:341] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:08.159114 27723 raft_consensus.cc:2454] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.159269 27388 leader_election.cc:302] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.159337 27846 raft_consensus.cc:2781] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.159384 27846 raft_consensus.cc:686] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.159423 27846 consensus_queue.cc:237] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.159940 27122 catalog_manager.cc:5484] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.160037 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:08.194348 27846 raft_consensus.cc:482] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.194432 27846 raft_consensus.cc:504] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.194540 27846 leader_election.cc:288] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.194679 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12a765f85d43f180a9ce0254c2fd10" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.194748 27723 raft_consensus.cc:2454] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:08.194797 27386 leader_election.cc:341] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:08.194914 27388 leader_election.cc:302] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.195025 27846 raft_consensus.cc:2781] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.195067 27846 raft_consensus.cc:482] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.195084 27846 raft_consensus.cc:3037] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.195704 27846 raft_consensus.cc:504] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.195816 27846 leader_election.cc:288] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.195923 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a12a765f85d43f180a9ce0254c2fd10" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.195987 27723 raft_consensus.cc:3037] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.196573 27723 raft_consensus.cc:2454] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
W20240503 02:09:08.196687 27386 leader_election.cc:341] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:08.196734 27388 leader_election.cc:302] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.196797 27846 raft_consensus.cc:2781] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.196841 27846 raft_consensus.cc:686] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.196882 27846 consensus_queue.cc:237] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.197455 27122 catalog_manager.cc:5484] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.200839 27904 raft_consensus.cc:482] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.200901 27904 raft_consensus.cc:504] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.201016 27904 leader_election.cc:288] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.201143 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aae0d9cd2ec4a7cb86ef48ef02f8ee6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.201207 27453 raft_consensus.cc:2454] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
W20240503 02:09:08.201251 27656 leader_election.cc:341] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:08.201352 27659 leader_election.cc:302] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.201431 27904 raft_consensus.cc:2781] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.201467 27904 raft_consensus.cc:482] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.201483 27904 raft_consensus.cc:3037] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.202028 27904 raft_consensus.cc:504] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.202139 27904 leader_election.cc:288] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.202279 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3aae0d9cd2ec4a7cb86ef48ef02f8ee6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.202337 27453 raft_consensus.cc:3037] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.202339 27904 raft_consensus.cc:482] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.202375 27904 raft_consensus.cc:504] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
W20240503 02:09:08.202436 27656 leader_election.cc:341] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:08.202441 27904 leader_election.cc:288] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.202572 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c11ab6468184080a8697375a119547c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.202595 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c11ab6468184080a8697375a119547c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.202641 27451 raft_consensus.cc:2454] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.202642 27587 raft_consensus.cc:2454] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.202749 27659 leader_election.cc:302] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.202817 27904 raft_consensus.cc:2781] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.202867 27904 raft_consensus.cc:482] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.202891 27904 raft_consensus.cc:3037] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.202946 27453 raft_consensus.cc:2454] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.203318 27904 raft_consensus.cc:504] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.203426 27904 leader_election.cc:288] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.203542 27659 leader_election.cc:302] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.203565 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c11ab6468184080a8697375a119547c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.203626 27453 raft_consensus.cc:3037] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.203629 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c11ab6468184080a8697375a119547c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.203697 27587 raft_consensus.cc:3037] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.204169 27453 raft_consensus.cc:2454] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.204289 27587 raft_consensus.cc:2454] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.204396 27659 leader_election.cc:302] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.204484 27853 raft_consensus.cc:2781] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.204526 27853 raft_consensus.cc:686] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.204553 27900 raft_consensus.cc:2781] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.204612 27900 raft_consensus.cc:686] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.204566 27853 consensus_queue.cc:237] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.204646 27900 consensus_queue.cc:237] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.205199 27122 catalog_manager.cc:5484] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.205348 27122 catalog_manager.cc:5484] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } }
W20240503 02:09:08.206000 27386 consensus_peers.cc:485] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 -> Peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Couldn't send request to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 092ad3ae09fe47bbbe473643060c43d0. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:08.206135 27723 raft_consensus.cc:1261] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:08.206504 27846 consensus_queue.cc:1035] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.207132 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:08.207479 27892 tablet.cc:1722] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.208469 27893 tablet.cc:1722] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.209935 27122 catalog_manager.cc:6029] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446] alter complete (version 2)
I20240503 02:09:08.212003 28054 raft_consensus.cc:482] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.212067 28054 raft_consensus.cc:504] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.212158 28054 leader_election.cc:288] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.212304 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dc7b64b7d4484296de6b8fc78e2705" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.212366 27587 raft_consensus.cc:2454] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.212510 27386 leader_election.cc:302] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.212477 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dc7b64b7d4484296de6b8fc78e2705" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.212550 27723 raft_consensus.cc:2454] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.212595 28054 raft_consensus.cc:2781] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.212651 28054 raft_consensus.cc:482] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.212683 28054 raft_consensus.cc:3037] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.213368 28054 raft_consensus.cc:504] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.213500 28054 leader_election.cc:288] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.213642 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dc7b64b7d4484296de6b8fc78e2705" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.213663 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42dc7b64b7d4484296de6b8fc78e2705" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.213703 27723 raft_consensus.cc:3037] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.213723 27587 raft_consensus.cc:3037] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.214232 27587 raft_consensus.cc:2454] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.214232 27723 raft_consensus.cc:2454] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.214427 27388 leader_election.cc:302] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.214519 28054 raft_consensus.cc:2781] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.214560 28054 raft_consensus.cc:686] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.214602 28054 consensus_queue.cc:237] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.215265 27124 catalog_manager.cc:5484] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.228449 28054 raft_consensus.cc:482] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.228506 28054 raft_consensus.cc:504] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.228607 28054 leader_election.cc:288] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.228750 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc8839c6cb14c648d15ef51ccb722ea" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.228753 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc8839c6cb14c648d15ef51ccb722ea" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.228816 27723 raft_consensus.cc:2454] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.228816 27587 raft_consensus.cc:2454] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:08.228935 27386 leader_election.cc:302] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.229018 28054 raft_consensus.cc:2781] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.229053 28054 raft_consensus.cc:482] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.229068 28054 raft_consensus.cc:3037] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.229580 28054 raft_consensus.cc:504] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.229707 28054 leader_election.cc:288] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.229820 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc8839c6cb14c648d15ef51ccb722ea" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.229844 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc8839c6cb14c648d15ef51ccb722ea" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.229883 27723 raft_consensus.cc:3037] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.229894 27587 raft_consensus.cc:3037] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.230425 27587 raft_consensus.cc:2454] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.230551 27386 leader_election.cc:302] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.230593 27723 raft_consensus.cc:2454] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.230608 28054 raft_consensus.cc:2781] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.230648 28054 raft_consensus.cc:686] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.230679 28054 consensus_queue.cc:237] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.231242 27124 catalog_manager.cc:5484] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.231524 28054 consensus_queue.cc:1035] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.233098 27852 consensus_queue.cc:1035] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.260078 28007 raft_consensus.cc:482] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.260144 28007 raft_consensus.cc:504] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.260246 28007 leader_election.cc:288] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:08.260483 27524 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
W20240503 02:09:08.260560 27523 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:08.260610 27523 leader_election.cc:302] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:08.260682 28007 raft_consensus.cc:2726] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:08.282599 27900 raft_consensus.cc:482] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.282673 27900 raft_consensus.cc:504] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.282784 27900 leader_election.cc:288] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.282882 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d59c77e3e1444b2fb211363d59bd0631" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.282920 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d59c77e3e1444b2fb211363d59bd0631" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.282950 27453 raft_consensus.cc:2454] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.282968 27587 raft_consensus.cc:2454] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.283106 27659 leader_election.cc:302] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.283188 27900 raft_consensus.cc:2781] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.283238 27900 raft_consensus.cc:482] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.283264 27900 raft_consensus.cc:3037] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.283833 27900 raft_consensus.cc:504] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.283955 27900 leader_election.cc:288] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:08.284051 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d59c77e3e1444b2fb211363d59bd0631" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.284112 27453 raft_consensus.cc:3037] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.284130 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d59c77e3e1444b2fb211363d59bd0631" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.284183 27587 raft_consensus.cc:3037] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.284687 27453 raft_consensus.cc:2454] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.284790 27587 raft_consensus.cc:2454] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.284845 27659 leader_election.cc:302] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.285015 27900 raft_consensus.cc:2781] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.285059 27900 raft_consensus.cc:686] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.285099 27900 consensus_queue.cc:237] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.285848 27124 catalog_manager.cc:5484] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.288244 28007 raft_consensus.cc:482] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.288306 28007 raft_consensus.cc:504] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.288427 28007 leader_election.cc:288] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.288898 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1e79e771df40f7abcdecaf683ae61b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.288959 27453 raft_consensus.cc:2454] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.288940 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1e79e771df40f7abcdecaf683ae61b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.289019 27723 raft_consensus.cc:2454] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.289139 27524 leader_election.cc:302] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.289268 28007 raft_consensus.cc:2781] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.289304 28007 raft_consensus.cc:482] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.289338 28007 raft_consensus.cc:3037] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.289904 28007 raft_consensus.cc:504] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.290019 28007 leader_election.cc:288] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.291010 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1e79e771df40f7abcdecaf683ae61b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.291086 27723 raft_consensus.cc:3037] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.291069 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b1e79e771df40f7abcdecaf683ae61b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.291124 27453 raft_consensus.cc:3037] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.291618 27723 raft_consensus.cc:2454] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.291635 27453 raft_consensus.cc:2454] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.291788 27523 leader_election.cc:302] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.292300 28007 raft_consensus.cc:2781] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.292354 28007 raft_consensus.cc:686] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:08.292389 28007 consensus_queue.cc:237] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.293668 28179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:09:08.293773 27124 catalog_manager.cc:5484] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.293874 27124 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.294755 28179 pending_rounds.cc:76] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Aborting all ops after (but not including) 4
I20240503 02:09:08.294797 28179 pending_rounds.cc:97] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Aborting uncommitted WRITE_OP operation due to leader change: 1.5
I20240503 02:09:08.294912 28179 pending_rounds.cc:97] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Aborting uncommitted WRITE_OP operation due to leader change: 1.6
I20240503 02:09:08.298887 28313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:08.298966 28313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:08.299099 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c324c9220fd14d3c817b18bbd3774439. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:08.299180 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:08.300134 27815 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:08.300204 27815 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:08.307807 28225 mvcc.cc:205] Tried to move back new op lower bound from 7023419999379578880 to 7023419994222440448. Current Snapshot: MvccSnapshot[applied={T|T < 7023419999155859456}]
I20240503 02:09:08.309165 27900 raft_consensus.cc:482] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.309227 27900 raft_consensus.cc:504] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.309350 27900 leader_election.cc:288] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.309454 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1fbd4e5618451fac90622fbcf1e9bb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.309484 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1fbd4e5618451fac90622fbcf1e9bb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:08.309513 27453 raft_consensus.cc:2454] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.309552 27587 raft_consensus.cc:2454] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:08.309639 27659 leader_election.cc:302] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.309718 27900 raft_consensus.cc:2781] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.309747 27900 raft_consensus.cc:482] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.309772 27900 raft_consensus.cc:3037] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.310310 27900 raft_consensus.cc:504] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.310422 27900 leader_election.cc:288] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.310524 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1fbd4e5618451fac90622fbcf1e9bb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:08.310587 27587 raft_consensus.cc:3037] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.310573 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1fbd4e5618451fac90622fbcf1e9bb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.310622 27453 raft_consensus.cc:3037] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.311028 27453 raft_consensus.cc:2454] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.311031 27587 raft_consensus.cc:2454] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:08.311167 27656 leader_election.cc:302] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.311252 27900 raft_consensus.cc:2781] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.311290 27900 raft_consensus.cc:686] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:08.311332 27900 consensus_queue.cc:237] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.311815 27124 catalog_manager.cc:5484] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:08.342432 27846 raft_consensus.cc:482] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.342485 27846 raft_consensus.cc:504] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.342599 27846 leader_election.cc:288] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.342732 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f54b533856464e81d77e726b2b7f31" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.342814 27723 raft_consensus.cc:2454] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:08.342842 27386 leader_election.cc:341] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:08.342949 27388 leader_election.cc:302] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.343024 27846 raft_consensus.cc:2781] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.343075 27846 raft_consensus.cc:482] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.343101 27846 raft_consensus.cc:3037] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.343538 27846 raft_consensus.cc:504] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.343638 27846 leader_election.cc:288] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:08.343750 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f54b533856464e81d77e726b2b7f31" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.343808 27723 raft_consensus.cc:3037] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:08.343863 27386 leader_election.cc:341] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:08.344261 27723 raft_consensus.cc:2454] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:08.344408 27388 leader_election.cc:302] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:08.344492 27846 raft_consensus.cc:2781] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.344537 27846 raft_consensus.cc:686] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:08.344578 27846 consensus_queue.cc:237] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.345191 27124 catalog_manager.cc:5484] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.362036 27853 consensus_queue.cc:1035] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.363389 27904 consensus_queue.cc:1035] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.365372 27884 raft_consensus.cc:482] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.365425 27884 raft_consensus.cc:504] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.365556 27884 leader_election.cc:288] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.365725 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.365758 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5443787e1d464baf93549b49571f5687" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.365823 27453 raft_consensus.cc:2379] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:08.365959 27524 leader_election.cc:302] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:08.366103 27884 raft_consensus.cc:3037] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.366693 27884 raft_consensus.cc:2726] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:08.412513 27884 consensus_queue.cc:1035] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.413766 27848 consensus_queue.cc:1035] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.420078 27848 raft_consensus.cc:482] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:08.420125 27848 raft_consensus.cc:504] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.420214 27848 leader_election.cc:288] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.420369 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3d3ca1b7ac478d8f5f942d6892d600" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:08.420377 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3d3ca1b7ac478d8f5f942d6892d600" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:08.420439 27453 raft_consensus.cc:2454] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.420439 27723 raft_consensus.cc:2454] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:08.420574 27523 leader_election.cc:302] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:08.420655 27848 raft_consensus.cc:2781] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:08.420707 27848 raft_consensus.cc:482] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:08.420725 27848 raft_consensus.cc:3037] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.421315 27848 raft_consensus.cc:504] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.421468 27848 leader_election.cc:288] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.421571 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3d3ca1b7ac478d8f5f942d6892d600" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:08.421624 27723 raft_consensus.cc:3037] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.421644 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd3d3ca1b7ac478d8f5f942d6892d600" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:08.421701 27453 raft_consensus.cc:3037] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:08.422138 27453 raft_consensus.cc:2454] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.422194 27723 raft_consensus.cc:2454] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:08.422294 27524 leader_election.cc:302] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:08.422370 27848 raft_consensus.cc:2781] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:08.422415 27848 raft_consensus.cc:686] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:08.422449 27848 consensus_queue.cc:237] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } }
I20240503 02:09:08.422935 27124 catalog_manager.cc:5484] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:08.436771 27848 consensus_queue.cc:1035] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.437547 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.438256 27124 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.438366 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.438469 27884 consensus_queue.cc:1035] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.439383 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.439851 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.459914 28054 consensus_queue.cc:1035] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.461485 28054 consensus_queue.cc:1035] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:08.464397 27524 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Couldn't send request to peer bd4b79bd8b57443faa6fa5e464570188. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 1: this message will repeat every 5th retry.
W20240503 02:09:08.504595 27523 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Couldn't send request to peer c2be543b8aad4e80a58d6b4ab669faae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:08.519115 27853 consensus_queue.cc:1035] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.520606 27904 consensus_queue.cc:1035] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.522383 27904 consensus_queue.cc:1035] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.523981 27904 consensus_queue.cc:1035] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.550915 28054 consensus_queue.cc:1035] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.552294 27852 consensus_queue.cc:1035] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.553654 27846 consensus_queue.cc:1035] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.555166 27846 consensus_queue.cc:1035] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.568807 27846 consensus_queue.cc:1035] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.569747 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.570281 27852 consensus_queue.cc:1035] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.570611 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.570793 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.571528 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.572000 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:08.577420 27848 consensus_queue.cc:1035] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.579094 28007 consensus_queue.cc:1035] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.581835 27904 consensus_queue.cc:1035] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.583005 27904 consensus_queue.cc:1035] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.632894 27846 consensus_queue.cc:1035] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.637925 27846 consensus_queue.cc:1035] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.662710 27904 consensus_queue.cc:1035] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.664052 27900 consensus_queue.cc:1035] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.665944 27846 consensus_queue.cc:1035] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.667312 27846 consensus_queue.cc:1035] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.677490 28007 consensus_queue.cc:1035] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.678812 27884 consensus_queue.cc:1035] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.681365 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.681463 27122 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.681906 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.681988 27124 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.683408 27124 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:08.688442 27904 consensus_queue.cc:1035] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.689707 27853 consensus_queue.cc:1035] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.690554 28352 ts_tablet_manager.cc:902] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Initiating tablet copy from peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:08.690953 27852 consensus_queue.cc:1035] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.691253 28352 tablet_copy_client.cc:308] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Beginning tablet copy session from remote peer at address 127.26.117.1:46597
I20240503 02:09:08.691859 27473 tablet_copy_service.cc:140] P bd4b79bd8b57443faa6fa5e464570188: Received BeginTabletCopySession request for tablet 092ad3ae09fe47bbbe473643060c43d0 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db ({username='slave'} at 127.26.117.2:44615)
I20240503 02:09:08.691918 27473 tablet_copy_service.cc:161] P bd4b79bd8b57443faa6fa5e464570188: Beginning new tablet copy session on tablet 092ad3ae09fe47bbbe473643060c43d0 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db at {username='slave'} at 127.26.117.2:44615: session id = 5cc801b4e0b047b3b4b30f8dd5c7c5db-092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:08.692589 27473 tablet_copy_source_session.cc:205] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:09:08.693671 28352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092ad3ae09fe47bbbe473643060c43d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:08.694499 27386 consensus_peers.cc:485] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 -> Peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Couldn't send request to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a12a765f85d43f180a9ce0254c2fd10. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:08.695655 28352 tablet_copy_client.cc:783] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 0 data blocks...
I20240503 02:09:08.695782 28352 tablet_copy_client.cc:647] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 1 WAL segments...
I20240503 02:09:08.697361 28352 tablet_copy_client.cc:523] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:09:08.698138 28352 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:08.699543 28352 tablet.cc:1722] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:08.699633 28352 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20240503 02:09:08.699743 28352 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:08.699911 28352 tablet_bootstrap.cc:492] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap complete.
I20240503 02:09:08.700654 28352 ts_tablet_manager.cc:1367] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240503 02:09:08.700796 28352 raft_consensus.cc:348] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.700855 28352 raft_consensus.cc:729] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:08.700887 28352 consensus_queue.cc:260] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:08.701112 28352 ts_tablet_manager.cc:1398] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:08.701617 27473 tablet_copy_service.cc:342] P bd4b79bd8b57443faa6fa5e464570188: Request end of tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-092ad3ae09fe47bbbe473643060c43d0 received from {username='slave'} at 127.26.117.2:44615
I20240503 02:09:08.701668 27473 tablet_copy_service.cc:433] P bd4b79bd8b57443faa6fa5e464570188: ending tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-092ad3ae09fe47bbbe473643060c43d0 on tablet 092ad3ae09fe47bbbe473643060c43d0 with peer 5cc801b4e0b047b3b4b30f8dd5c7c5db
W20240503 02:09:08.725263 27656 consensus_peers.cc:485] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae -> Peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Couldn't send request to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3aae0d9cd2ec4a7cb86ef48ef02f8ee6. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:08.735517 28054 consensus_queue.cc:1035] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.736873 27852 consensus_queue.cc:1035] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.766985 27853 consensus_queue.cc:1035] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:08.792491 27386 consensus_peers.cc:485] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 -> Peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Couldn't send request to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4f54b533856464e81d77e726b2b7f31. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:08.858942 27852 consensus_queue.cc:1035] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.861377 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:08.880111 27848 consensus_queue.cc:1035] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.881451 28007 consensus_queue.cc:1035] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:08.924175 28369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:08.924242 28369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:08.924438 28369 leader_election.cc:288] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:08.927517 28179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 296 } } ignore_live_leader: false dest_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" is_pre_election: true
W20240503 02:09:09.026841 27122 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:09.085755 27582 raft_consensus.cc:1203] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.2] Dedup: 1.2->[]
I20240503 02:09:09.123452 27122 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
W20240503 02:09:09.160401 27118 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
I20240503 02:09:09.188227 28352 ts_tablet_manager.cc:902] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Initiating tablet copy from peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:09.188489 28352 tablet_copy_client.cc:308] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Beginning tablet copy session from remote peer at address 127.26.117.1:46597
I20240503 02:09:09.188694 27473 tablet_copy_service.cc:140] P bd4b79bd8b57443faa6fa5e464570188: Received BeginTabletCopySession request for tablet f4f54b533856464e81d77e726b2b7f31 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db ({username='slave'} at 127.26.117.2:44615)
I20240503 02:09:09.188759 27473 tablet_copy_service.cc:161] P bd4b79bd8b57443faa6fa5e464570188: Beginning new tablet copy session on tablet f4f54b533856464e81d77e726b2b7f31 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db at {username='slave'} at 127.26.117.2:44615: session id = 5cc801b4e0b047b3b4b30f8dd5c7c5db-f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:09.189276 27473 tablet_copy_source_session.cc:205] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:09:09.189611 28352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f54b533856464e81d77e726b2b7f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:09.190734 28352 tablet_copy_client.cc:783] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 0 data blocks...
I20240503 02:09:09.190840 28352 tablet_copy_client.cc:647] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 1 WAL segments...
I20240503 02:09:09.191334 28352 tablet_copy_client.cc:523] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:09:09.191995 28352 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:09.193003 28352 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:09.193167 28352 tablet_bootstrap.cc:492] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap complete.
I20240503 02:09:09.193244 28352 ts_tablet_manager.cc:1367] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:09.193379 28352 raft_consensus.cc:348] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.193442 28352 raft_consensus.cc:729] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:09.193485 28352 consensus_queue.cc:260] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.193581 28352 ts_tablet_manager.cc:1398] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:09.194000 27473 tablet_copy_service.cc:342] P bd4b79bd8b57443faa6fa5e464570188: Request end of tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-f4f54b533856464e81d77e726b2b7f31 received from {username='slave'} at 127.26.117.2:44615
I20240503 02:09:09.194048 27473 tablet_copy_service.cc:433] P bd4b79bd8b57443faa6fa5e464570188: ending tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-f4f54b533856464e81d77e726b2b7f31 on tablet f4f54b533856464e81d77e726b2b7f31 with peer 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:09.222129 27122 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:09.276309 28352 ts_tablet_manager.cc:902] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Initiating tablet copy from peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:09.276583 28352 tablet_copy_client.cc:308] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Beginning tablet copy session from remote peer at address 127.26.117.1:46597
I20240503 02:09:09.276830 27473 tablet_copy_service.cc:140] P bd4b79bd8b57443faa6fa5e464570188: Received BeginTabletCopySession request for tablet 3a12a765f85d43f180a9ce0254c2fd10 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db ({username='slave'} at 127.26.117.2:44615)
I20240503 02:09:09.276894 27473 tablet_copy_service.cc:161] P bd4b79bd8b57443faa6fa5e464570188: Beginning new tablet copy session on tablet 3a12a765f85d43f180a9ce0254c2fd10 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db at {username='slave'} at 127.26.117.2:44615: session id = 5cc801b4e0b047b3b4b30f8dd5c7c5db-3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:09.277451 27473 tablet_copy_source_session.cc:205] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:09:09.277771 28352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a12a765f85d43f180a9ce0254c2fd10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:09.278934 28352 tablet_copy_client.cc:783] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 0 data blocks...
I20240503 02:09:09.279037 28352 tablet_copy_client.cc:647] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 1 WAL segments...
I20240503 02:09:09.279590 28352 tablet_copy_client.cc:523] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:09:09.280242 28352 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:09.281198 28352 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:09.281373 28352 tablet_bootstrap.cc:492] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap complete.
I20240503 02:09:09.281450 28352 ts_tablet_manager.cc:1367] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:09.281582 28352 raft_consensus.cc:348] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.281646 28352 raft_consensus.cc:729] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:09.281688 28352 consensus_queue.cc:260] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.281801 28352 ts_tablet_manager.cc:1398] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:09.282217 27473 tablet_copy_service.cc:342] P bd4b79bd8b57443faa6fa5e464570188: Request end of tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-3a12a765f85d43f180a9ce0254c2fd10 received from {username='slave'} at 127.26.117.2:44615
I20240503 02:09:09.282265 27473 tablet_copy_service.cc:433] P bd4b79bd8b57443faa6fa5e464570188: ending tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-3a12a765f85d43f180a9ce0254c2fd10 on tablet 3a12a765f85d43f180a9ce0254c2fd10 with peer 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:09.315244 28352 ts_tablet_manager.cc:902] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Initiating tablet copy from peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:09.315469 28352 tablet_copy_client.cc:308] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Beginning tablet copy session from remote peer at address 127.26.117.3:35961
I20240503 02:09:09.315721 27743 tablet_copy_service.cc:140] P c2be543b8aad4e80a58d6b4ab669faae: Received BeginTabletCopySession request for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db ({username='slave'} at 127.26.117.2:49739)
I20240503 02:09:09.315785 27743 tablet_copy_service.cc:161] P c2be543b8aad4e80a58d6b4ab669faae: Beginning new tablet copy session on tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 from peer 5cc801b4e0b047b3b4b30f8dd5c7c5db at {username='slave'} at 127.26.117.2:49739: session id = 5cc801b4e0b047b3b4b30f8dd5c7c5db-3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:09.316471 27743 tablet_copy_source_session.cc:205] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:09:09.316857 28352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:09.317972 28352 tablet_copy_client.cc:783] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 0 data blocks...
I20240503 02:09:09.318073 28352 tablet_copy_client.cc:647] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Starting download of 1 WAL segments...
I20240503 02:09:09.318698 28352 tablet_copy_client.cc:523] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:09:09.319330 28352 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:09.320499 28352 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:09.320657 28352 tablet_bootstrap.cc:492] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap complete.
I20240503 02:09:09.320734 28352 ts_tablet_manager.cc:1367] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:09.320883 28352 raft_consensus.cc:348] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.320942 28352 raft_consensus.cc:729] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:09.320987 28352 consensus_queue.cc:260] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.321079 28352 ts_tablet_manager.cc:1398] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:09.321574 27743 tablet_copy_service.cc:342] P c2be543b8aad4e80a58d6b4ab669faae: Request end of tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-3aae0d9cd2ec4a7cb86ef48ef02f8ee6 received from {username='slave'} at 127.26.117.2:49739
I20240503 02:09:09.321627 27743 tablet_copy_service.cc:433] P c2be543b8aad4e80a58d6b4ab669faae: ending tablet copy session 5cc801b4e0b047b3b4b30f8dd5c7c5db-3aae0d9cd2ec4a7cb86ef48ef02f8ee6 on tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 with peer 5cc801b4e0b047b3b4b30f8dd5c7c5db
W20240503 02:09:09.375896 27122 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:09.444774 27122 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:09.540433 27884 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:09.540516 27884 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.540643 27884 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:09.540896 27524 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
W20240503 02:09:09.540985 27523 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
I20240503 02:09:09.541059 27523 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:09.541123 27884 raft_consensus.cc:2726] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:09.719842 27582 raft_consensus.cc:1203] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20240503 02:09:09.799240 27582 raft_consensus.cc:1203] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20240503 02:09:09.828722 27582 raft_consensus.cc:1203] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20240503 02:09:09.909780 27884 raft_consensus.cc:482] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:09.909848 27884 raft_consensus.cc:504] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:09.909969 27884 leader_election.cc:288] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:09.910214 27524 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
W20240503 02:09:09.910264 27523 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:09.910296 27523 leader_election.cc:302] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:09.910367 27884 raft_consensus.cc:2726] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20240503 02:09:09.929534 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:09.947777 28399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:09.947885 28399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:09.948173 28399 leader_election.cc:288] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847)
I20240503 02:09:09.948323 27140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "c324c9220fd14d3c817b18bbd3774439" is_pre_election: true
I20240503 02:09:09.948477 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" is_pre_election: true
I20240503 02:09:09.948673 28147 leader_election.cc:302] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1; no voters: 9e12d823f5eb49f38d5fdc7ee2185bb4, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:09.948796 28399 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240503 02:09:10.011188 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:10.160060 27848 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:10.160138 27848 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:10.160270 27848 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:10.160524 27524 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
W20240503 02:09:10.160600 27523 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:10.160653 27523 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:10.160715 27848 raft_consensus.cc:2726] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240503 02:09:10.193504 27118 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
I20240503 02:09:10.299130 27848 raft_consensus.cc:482] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:10.299208 27848 raft_consensus.cc:504] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:10.299336 27848 leader_election.cc:288] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:10.299551 27524 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
W20240503 02:09:10.299618 27523 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:10.299667 27523 leader_election.cc:302] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:10.299736 27848 raft_consensus.cc:2726] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240503 02:09:10.375190 27188 leader_election.cc:334] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Timed out: RequestConsensusVote RPC to 127.26.117.62:37529 timed out after 0.724s (SENT)
I20240503 02:09:10.375260 27188 leader_election.cc:302] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters: 9c75a5f0d0824fafbc14d2a2c1613db1, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:10.375442 28401 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240503 02:09:10.382793 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:10.383793 27092 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpsd1k2/build/release/bin/kudu with pid 28089
W20240503 02:09:10.493569 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:10.642313 27118 consensus_peers.cc:485] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 -> Peer 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369): Couldn't send request to peer 9c75a5f0d0824fafbc14d2a2c1613db1. Status: Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240503 02:09:10.891168 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:10.984576 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:11.030727 28413 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:11.030833 28413 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:11.136338 28415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:11.136421 28415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:11.136559 28415 leader_election.cc:288] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
W20240503 02:09:11.137082 27190 leader_election.cc:334] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369): Network error: Client connection negotiation failed: client connection to 127.26.117.60:36369: connect: Connection refused (error 111)
W20240503 02:09:11.350958 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:11.369789 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--webserver_interface=127.26.117.60
--webserver_port=44717
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20240503 02:09:11.465761 28420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:11.465947 28420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:11.465970 28420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:11.467904 28420 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:11.467947 28420 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:11.467962 28420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:11.467976 28420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:11.467999 28420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:11.468024 28420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:11.469388 28420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.60:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.117.60
--webserver_port=44717
--never_fsync=true
--heap_profile_path=/tmp/kudu.28420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:11.469568 28420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:11.469785 28420 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:11.474156 28425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:11.474561 28426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:11.474946 28428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:11.475315 28420 server_base.cc:1034] running on GCE node
I20240503 02:09:11.475523 28420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:11.475771 28420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:11.476899 28420 hybrid_clock.cc:648] HybridClock initialized: now 1714702151476885 us; error 26 us; skew 500 ppm
I20240503 02:09:11.481313 28420 webserver.cc:460] Webserver started at http://127.26.117.60:44717/ using document root <none> and password file <none>
I20240503 02:09:11.481519 28420 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:11.481563 28420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:11.482856 28420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:11.483457 28434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:11.483635 28420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:11.483707 28420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:11.483949 28420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:11.500779 28420 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:11.501008 28420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:11.501098 28420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:11.505014 28420 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.60:36369
I20240503 02:09:11.505048 28486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.60:36369 every 8 connection(s)
I20240503 02:09:11.505380 28420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-2/data/info.pb
I20240503 02:09:11.506942 28487 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:09:11.507452 28487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap starting.
I20240503 02:09:11.511677 28487 log.cc:824] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:11.514495 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 28420
I20240503 02:09:11.518942 27848 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:11.519008 27848 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:11.519137 27848 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:11.519413 27524 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
W20240503 02:09:11.519495 27523 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
I20240503 02:09:11.519544 27523 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:11.519618 27848 raft_consensus.cc:2726] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:11.528712 28487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap replayed 1/1 log segments. Stats: ops{read=304 overwritten=2 applied=302 ignored=0} inserts{seen=150 ignored=0} mutations{seen=363 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:11.528975 28487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: Bootstrap complete.
I20240503 02:09:11.530468 28487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:11.530699 28487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c75a5f0d0824fafbc14d2a2c1613db1, State: Initialized, Role: FOLLOWER
I20240503 02:09:11.530810 28487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 302, Last appended: 2.302, Last appended by leader: 302, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:11.531064 28487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:11.531034 28492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:11.531236 28492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:11.532987 28503 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:11.533035 28503 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: loading cluster ID for follower catalog manager: success
I20240503 02:09:11.534370 28503 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: acquiring CA information for follower catalog manager: success
I20240503 02:09:11.534758 28503 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:11.578205 28492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c324c9220fd14d3c817b18bbd3774439. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:11.578289 28492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
W20240503 02:09:11.582070 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:11.583916 27634 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:11.584796 28451 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" instance_seqno: 1714702146750084) as {username='slave'} at 127.26.117.2:43287; Asking this server to re-register.
I20240503 02:09:11.585048 27634 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:11.585498 28451 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:11.725656 27769 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:11.726047 28451 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" instance_seqno: 1714702146863903) as {username='slave'} at 127.26.117.3:46625; Asking this server to re-register.
I20240503 02:09:11.726241 27769 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:11.726588 28451 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:11.832383 27124 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:11.835047 27499 heartbeater.cc:347] Connected to a master server at 127.26.117.60:36369
I20240503 02:09:11.835515 28451 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" instance_seqno: 1714702146630581) as {username='slave'} at 127.26.117.1:41753; Asking this server to re-register.
I20240503 02:09:11.835822 27499 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:11.836150 28451 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
W20240503 02:09:11.842167 27188 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.117.62:37529 timed out after 0.705s (SENT) [suppressed 3 similar messages]
W20240503 02:09:11.867363 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:11.868005 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:11.868072 27124 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:11.868131 27124 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]). Expected version 1 got 0
I20240503 02:09:11.948738 28410 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:11.948804 28410 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:11.948936 28410 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:11.949193 27524 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
W20240503 02:09:11.949247 27523 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:11.949312 27523 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:11.949422 28410 raft_consensus.cc:2726] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240503 02:09:12.016523 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:12.078899 28514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:12.079005 28514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:12.079277 28514 leader_election.cc:288] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847)
I20240503 02:09:12.082623 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" is_pre_election: true
I20240503 02:09:12.131820 28413 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c324c9220fd14d3c817b18bbd3774439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:12.131920 28413 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:12.168300 27848 raft_consensus.cc:482] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:12.168359 27848 raft_consensus.cc:504] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:12.168474 27848 leader_election.cc:288] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:12.168704 27523 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
W20240503 02:09:12.168763 27524 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:12.168799 27524 leader_election.cc:302] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:12.168874 27848 raft_consensus.cc:2726] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:12.213477 27848 raft_consensus.cc:482] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:12.213524 27848 raft_consensus.cc:504] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:12.213629 27848 leader_election.cc:288] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:12.213872 27524 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
W20240503 02:09:12.213943 27523 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:12.214004 27523 leader_election.cc:302] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:12.214068 27848 raft_consensus.cc:2726] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240503 02:09:12.258608 27118 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 8127ac16597e40fa9b1590a61430aea0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8127ac16597e40fa9b1590a61430aea0 consensus state has no leader
W20240503 02:09:12.267211 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:12.476253 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:12.548249 27188 leader_election.cc:334] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Timed out: RequestConsensusVote RPC to 127.26.117.62:37529 timed out after 0.705s (SENT)
I20240503 02:09:12.548318 27188 leader_election.cc:302] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters: 9c75a5f0d0824fafbc14d2a2c1613db1, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:12.548470 28520 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240503 02:09:12.788800 28436 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.26.117.62:37529 for RPC RequestConsensusVote timed out after 0.708s (ON_OUTBOUND_QUEUE)
W20240503 02:09:12.834070 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:12.964697 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:13.109196 28524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:13.109279 28524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:13.109429 28524 leader_election.cc:288] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:13.112605 28461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" is_pre_election: true
W20240503 02:09:13.418095 27524 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Couldn't send request to peer bd4b79bd8b57443faa6fa5e464570188. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 6: this message will repeat every 5th retry.
W20240503 02:09:13.418159 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:13.497646 28436 leader_election.cc:334] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Timed out: connection negotiation to 127.26.117.62:37529 for RPC RequestConsensusVote timed out after 0.708s (ON_OUTBOUND_QUEUE)
I20240503 02:09:13.497726 28436 leader_election.cc:302] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1; no voters: 9e12d823f5eb49f38d5fdc7ee2185bb4, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:13.497924 28527 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240503 02:09:13.545933 27523 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Couldn't send request to peer c2be543b8aad4e80a58d6b4ab669faae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 6: this message will repeat every 5th retry.
W20240503 02:09:13.545941 27124 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:13.688262 27092 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpsd1k2/build/release/bin/kudu with pid 27100
W20240503 02:09:13.692267 27777 connection.cc:535] client connection to 127.26.117.62:37529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:09:13.692369 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 02:09:11.532850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.117.62:37529 (local address 127.0.0.1:41838)
0503 02:09:11.537522 (+ 4672us) negotiation.cc:107] Waiting for socket to connect
0503 02:09:11.537545 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0503 02:09:11.537607 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0503 02:09:13.692227 (+2154620us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":4649,"thread_start_us":75,"threads_started":1}
W20240503 02:09:13.692457 27777 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.26.117.62:37529)
W20240503 02:09:13.692538 27777 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.26.117.62:37529)
W20240503 02:09:13.692567 27777 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.117.62:37529)
W20240503 02:09:13.692613 27777 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.26.117.62:37529)
W20240503 02:09:13.692612 27188 connection.cc:535] client connection to 127.26.117.62:37529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:09:13.692977 27524 connection.cc:535] server connection from 127.0.0.1:42354 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240503 02:09:13.692803 28420 master_runner.cc:186] Error getting master registration for 127.26.117.62:37529: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20240503 02:09:13.694494 28420 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20240503 02:09:13.693203 27172 master_runner.cc:186] Error getting master registration for 127.26.117.62:37529: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107),
I20240503 02:09:13.694911 27172 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20240503 02:09:13.695196 27188 leader_election.cc:334] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection refused (error 111)
I20240503 02:09:13.695245 27188 leader_election.cc:302] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters: 9c75a5f0d0824fafbc14d2a2c1613db1, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:13.695372 28533 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240503 02:09:13.928179 27848 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:13.928242 27848 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:13.928373 27848 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:13.928611 27523 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
W20240503 02:09:13.928674 27524 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:13.928718 27524 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:13.928782 27848 raft_consensus.cc:2726] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:14.016394 28539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:14.016456 28539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.016548 28539 leader_election.cc:288] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9e12d823f5eb49f38d5fdc7ee2185bb4 (127.26.117.61:37847)
I20240503 02:09:14.016705 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" is_pre_election: true
W20240503 02:09:14.017035 28436 leader_election.cc:334] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection refused (error 111)
I20240503 02:09:14.017079 28436 leader_election.cc:302] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1; no voters: 9e12d823f5eb49f38d5fdc7ee2185bb4, c324c9220fd14d3c817b18bbd3774439
I20240503 02:09:14.017138 28539 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20240503 02:09:14.236975 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--webserver_interface=127.26.117.62
--webserver_port=40493
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20240503 02:09:14.313072 28542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:14.313228 28542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:14.313251 28542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:09:14.314570 28542 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:14.314612 28542 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:14.314630 28542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:14.314644 28542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:14.314659 28542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:14.314675 28542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:14.316012 28542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.117.62
--webserver_port=40493
--never_fsync=true
--heap_profile_path=/tmp/kudu.28542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:14.316190 28542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:14.316370 28542 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:09:14.319777 28548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:14.320137 28550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:14.320525 28547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:14.320938 28542 server_base.cc:1034] running on GCE node
I20240503 02:09:14.321117 28542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:14.321300 28542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:14.322461 28542 hybrid_clock.cc:648] HybridClock initialized: now 1714702154322453 us; error 28 us; skew 500 ppm
I20240503 02:09:14.323489 28542 webserver.cc:460] Webserver started at http://127.26.117.62:40493/ using document root <none> and password file <none>
I20240503 02:09:14.323648 28542 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:14.323714 28542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:14.324707 28542 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:14.325202 28556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.325361 28542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.325433 28542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
uuid: "c324c9220fd14d3c817b18bbd3774439"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:14.325645 28542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:14.342239 27848 raft_consensus.cc:482] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.342296 27848 raft_consensus.cc:504] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.342417 27848 leader_election.cc:288] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:14.342626 27523 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
W20240503 02:09:14.342707 27524 leader_election.cc:341] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 8127ac16597e40fa9b1590a61430aea0
I20240503 02:09:14.342762 27524 leader_election.cc:302] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:14.342829 27848 raft_consensus.cc:2726] T 8127ac16597e40fa9b1590a61430aea0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:14.354571 28542 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:14.354769 28542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:14.354852 28542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:14.357946 28542 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.62:37529
I20240503 02:09:14.357971 28609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.62:37529 every 8 connection(s)
I20240503 02:09:14.358227 28542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
I20240503 02:09:14.359468 28610 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:09:14.359937 28610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap starting.
I20240503 02:09:14.361441 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 28542
I20240503 02:09:14.364606 28610 log.cc:824] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:14.367849 28615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:14.367924 28615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.368079 28615 leader_election.cc:288] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:14.368242 28461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" is_pre_election: true
I20240503 02:09:14.368333 28461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e12d823f5eb49f38d5fdc7ee2185bb4 in term 2.
I20240503 02:09:14.368523 27190 leader_election.cc:302] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1, 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters:
I20240503 02:09:14.368623 28615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240503 02:09:14.368690 28615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c324c9220fd14d3c817b18bbd3774439)
I20240503 02:09:14.368719 28615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:09:14.369468 28615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.369594 28615 leader_election.cc:288] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 election: Requested vote from peers c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529), 9c75a5f0d0824fafbc14d2a2c1613db1 (127.26.117.60:36369)
I20240503 02:09:14.369726 28461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1"
I20240503 02:09:14.369805 28461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:09:14.370915 28461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e12d823f5eb49f38d5fdc7ee2185bb4 in term 3.
I20240503 02:09:14.371052 27190 leader_election.cc:302] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c75a5f0d0824fafbc14d2a2c1613db1, 9e12d823f5eb49f38d5fdc7ee2185bb4; no voters:
I20240503 02:09:14.371147 28615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:09:14.371271 28615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [term 3 LEADER]: Becoming Leader. State: Replica: 9e12d823f5eb49f38d5fdc7ee2185bb4, State: Running, Role: LEADER
I20240503 02:09:14.371379 28615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 302, Committed index: 302, Last appended: 2.302, Last appended by leader: 302, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.371614 28615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e12d823f5eb49f38d5fdc7ee2185bb4. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.371665 28615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:14.372326 28619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:09:14.372753 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641]
I20240503 02:09:14.373023 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]
I20240503 02:09:14.373087 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e [id=0e67e01150414cfda8f556738345b4be]
I20240503 02:09:14.373136 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]
I20240503 02:09:14.373183 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]
I20240503 02:09:14.373226 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]
I20240503 02:09:14.373309 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]
I20240503 02:09:14.373387 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]
I20240503 02:09:14.373462 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_90f372f788f64dd0af4c4b9718ade63a [id=3383f9af04da490a8cb6f949d3d92644]
I20240503 02:09:14.373510 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6]
I20240503 02:09:14.373584 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a]
I20240503 02:09:14.373631 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]
I20240503 02:09:14.373689 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02]
I20240503 02:09:14.373736 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]
I20240503 02:09:14.373776 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]
I20240503 02:09:14.373847 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_dc8099703c974c61a042d8d2c291a023 [id=5f0720291c48410aa78697c47dd2831b]
I20240503 02:09:14.373904 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]
I20240503 02:09:14.373946 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]
I20240503 02:09:14.373986 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]
I20240503 02:09:14.374037 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]
I20240503 02:09:14.374078 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]
I20240503 02:09:14.374121 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]
I20240503 02:09:14.374159 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]
I20240503 02:09:14.374230 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc]
I20240503 02:09:14.374279 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]
I20240503 02:09:14.374344 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d]
I20240503 02:09:14.374411 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_0bfcd325883a414484aec17b7af9cb2f [id=c46a0a1c65dd47e38382f2b2d19bff05]
I20240503 02:09:14.374454 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]
I20240503 02:09:14.374493 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]
I20240503 02:09:14.374536 28619 catalog_manager.cc:669] Loaded metadata for table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]
I20240503 02:09:14.374827 28619 tablet_loader.cc:96] loaded metadata for tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446])
I20240503 02:09:14.374898 28619 tablet_loader.cc:96] loaded metadata for tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90f372f788f64dd0af4c4b9718ade63a [id=3383f9af04da490a8cb6f949d3d92644])
I20240503 02:09:14.374943 28619 tablet_loader.cc:96] loaded metadata for tablet 2ad4f6e1ac014739b60a30111b64a7c1 (table default.Table_d1dccb2cfe654cce8410d7503dbf8985 [id=41e048c6e5644782afa7023a8a808b02])
I20240503 02:09:14.375031 28619 tablet_loader.cc:96] loaded metadata for tablet 2ca063a19a044e6c8466598f44316ff5 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9])
I20240503 02:09:14.375083 28619 tablet_loader.cc:96] loaded metadata for tablet 388e82752def4a31b3d8bec18a478273 (table default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6])
I20240503 02:09:14.375156 28619 tablet_loader.cc:96] loaded metadata for tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3])
I20240503 02:09:14.375202 28619 tablet_loader.cc:96] loaded metadata for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (table default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42])
I20240503 02:09:14.375241 28619 tablet_loader.cc:96] loaded metadata for tablet 3a12a765f85d43f180a9ce0254c2fd10 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481])
I20240503 02:09:14.375278 28619 tablet_loader.cc:96] loaded metadata for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (table default.Table_059540465a8f493596bbda1239ae26b5 [id=04a56dd0fac64f1a845e5603e39fc641])
I20240503 02:09:14.375317 28619 tablet_loader.cc:96] loaded metadata for tablet 42dc7b64b7d4484296de6b8fc78e2705 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e])
I20240503 02:09:14.375355 28619 tablet_loader.cc:96] loaded metadata for tablet 442400993f8e43b79e429758d1f3c72f (table default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338])
I20240503 02:09:14.375418 28619 tablet_loader.cc:96] loaded metadata for tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_c839b3b37e0f40c487bdc964e6a6a999 [id=3995d190344c4d3f9d444cb266a9988a])
I20240503 02:09:14.375460 28619 tablet_loader.cc:96] loaded metadata for tablet 5443787e1d464baf93549b49571f5687 (table default.Table_0bfcd325883a414484aec17b7af9cb2f [id=c46a0a1c65dd47e38382f2b2d19bff05])
I20240503 02:09:14.375488 28619 tablet_loader.cc:96] loaded metadata for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_f54f404aa16e495d994c597b7ea98869 [id=b2a8365635134977a758514aa7a4351d])
I20240503 02:09:14.375524 28619 tablet_loader.cc:96] loaded metadata for tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66])
I20240503 02:09:14.375594 28619 tablet_loader.cc:96] loaded metadata for tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530])
I20240503 02:09:14.375639 28619 tablet_loader.cc:96] loaded metadata for tablet 9614b6572dfd443387ee3e4d676e8766 (table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc])
I20240503 02:09:14.375725 28619 tablet_loader.cc:96] loaded metadata for tablet 9c11ab6468184080a8697375a119547c (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e])
I20240503 02:09:14.375772 28619 tablet_loader.cc:96] loaded metadata for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_dc8099703c974c61a042d8d2c291a023 [id=5f0720291c48410aa78697c47dd2831b])
I20240503 02:09:14.375813 28619 tablet_loader.cc:96] loaded metadata for tablet b004b105a74e408380a4fe15bacbff03 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831])
I20240503 02:09:14.375850 28619 tablet_loader.cc:96] loaded metadata for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4])
I20240503 02:09:14.375921 28619 tablet_loader.cc:96] loaded metadata for tablet bdc8839c6cb14c648d15ef51ccb722ea (table default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b])
I20240503 02:09:14.375965 28619 tablet_loader.cc:96] loaded metadata for tablet ccd6dc9cfb834653ae08c29ff0f4848d (table default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e [id=0e67e01150414cfda8f556738345b4be])
I20240503 02:09:14.376009 28619 tablet_loader.cc:96] loaded metadata for tablet d59c77e3e1444b2fb211363d59bd0631 (table default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a])
I20240503 02:09:14.376129 28619 tablet_loader.cc:96] loaded metadata for tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd])
I20240503 02:09:14.376174 28619 tablet_loader.cc:96] loaded metadata for tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d])
I20240503 02:09:14.376214 28619 tablet_loader.cc:96] loaded metadata for tablet dd3d3ca1b7ac478d8f5f942d6892d600 (table default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f])
I20240503 02:09:14.376329 28619 tablet_loader.cc:96] loaded metadata for tablet e353654920a44994ae4889144db1b836 (table default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391])
I20240503 02:09:14.376381 28619 tablet_loader.cc:96] loaded metadata for tablet e43172b357ce45babdd7a4cfc2b8917a (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e])
I20240503 02:09:14.376441 28619 tablet_loader.cc:96] loaded metadata for tablet f4f54b533856464e81d77e726b2b7f31 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0])
I20240503 02:09:14.376482 28619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:09:14.376695 28619 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:14.376739 28619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:09:14.377063 28619 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:09:14.377279 28619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Loaded TSK: 0
I20240503 02:09:14.377481 28619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:09:14.383587 28610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap replayed 1/1 log segments. Stats: ops{read=302 overwritten=0 applied=302 ignored=0} inserts{seen=150 ignored=0} mutations{seen=363 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:14.383888 28610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap complete.
I20240503 02:09:14.383991 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet dd3d3ca1b7ac478d8f5f942d6892d600 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.384126 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_29c62440de5041adb7dc3747d41924ee" } modify_external_catalogs: true
I20240503 02:09:14.384215 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_16a939c3436b4c0fabceb3838811d215" } modify_external_catalogs: true
I20240503 02:09:14.384253 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_16a939c3436b4c0fabceb3838811d215" } modify_external_catalogs: true
I20240503 02:09:14.384161 27199 master_service.cc:946] ReplaceTablet: received request to replace tablet 39ac26cca1be4b7e8cbfa227ab39e38f from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.384193 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b795b4876be143edb345a11ccda4a55b" } modify_external_catalogs: true
I20240503 02:09:14.384538 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b795b4876be143edb345a11ccda4a55b" } modify_external_catalogs: true
I20240503 02:09:14.384586 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_32834d5c492a480090b2c41a0497a60f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.384759 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32834d5c492a480090b2c41a0497a60f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.384951 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bf8b557796624a6b8a7cce5936afa6ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.385036 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf8b557796624a6b8a7cce5936afa6ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.384142 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 89440f04f9d349a187c296efab0816d2 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.384124 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet e43172b357ce45babdd7a4cfc2b8917a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.384254 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_29c62440de5041adb7dc3747d41924ee" } modify_external_catalogs: true
I20240503 02:09:14.385557 27203 master_service.cc:946] ReplaceTablet: received request to replace tablet b004b105a74e408380a4fe15bacbff03 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.386147 28461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [term 3 FOLLOWER]: Refusing update from remote peer 9e12d823f5eb49f38d5fdc7ee2185bb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 302. Preceding OpId from leader: term: 3 index: 304. (index mismatch)
I20240503 02:09:14.386358 28617 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 303, Last known committed idx: 302, Time since last communication: 0.000s
I20240503 02:09:14.386492 28610 raft_consensus.cc:348] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.386742 28610 raft_consensus.cc:729] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c324c9220fd14d3c817b18bbd3774439, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.386879 28610 consensus_queue.cc:260] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 302, Last appended: 2.302, Last appended by leader: 302, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:14.387187 28610 sys_catalog.cc:564] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:14.387154 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.387266 28582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:09:14.387279 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:14.387566 28539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e12d823f5eb49f38d5fdc7ee2185bb4. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.387640 28539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:14.387789 28584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "c324c9220fd14d3c817b18bbd3774439" is_pre_election: true
I20240503 02:09:14.388235 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet dd3d3ca1b7ac478d8f5f942d6892d600
I20240503 02:09:14.388497 28582 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 3 FOLLOWER]: Refusing update from remote peer 9e12d823f5eb49f38d5fdc7ee2185bb4: Log matching property violated. Preceding OpId in replica: term: 2 index: 302. Preceding OpId from leader: term: 3 index: 304. (index mismatch)
I20240503 02:09:14.388520 27202 catalog_manager.cc:6554] ReplaceTablet: tablet dd3d3ca1b7ac478d8f5f942d6892d600 deleted and replaced by tablet 2e99d982dddd497988d2d7220b96aefa
I20240503 02:09:14.388590 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f553462ad4ea48fdb713726340f763d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.388664 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f553462ad4ea48fdb713726340f763d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.388806 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d59c77e3e1444b2fb211363d59bd0631
I20240503 02:09:14.388911 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a58c505992a54fdfa6be0c230dd83adb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.388990 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a58c505992a54fdfa6be0c230dd83adb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.388983 28539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.389057 28539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c75a5f0d0824fafbc14d2a2c1613db1 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:14.389102 28615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 303, Last known committed idx: 302, Time since last communication: 0.000s
I20240503 02:09:14.388984 28622 mvcc.cc:205] Tried to move back new op lower bound from 7023420024366137344 to 7023420024305721344. Current Snapshot: MvccSnapshot[applied={T|T < 7023420024364392448}]
I20240503 02:09:14.389377 27199 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 39ac26cca1be4b7e8cbfa227ab39e38f
I20240503 02:09:14.389444 27199 catalog_manager.cc:6554] ReplaceTablet: tablet 39ac26cca1be4b7e8cbfa227ab39e38f deleted and replaced by tablet 56b36a3207ff43c185993f0e9ea457fa
I20240503 02:09:14.389493 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_df038b49f4354dc0ba3846a1f0abe3df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.389564 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df038b49f4354dc0ba3846a1f0abe3df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.388952 28583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" candidate_term: 3 candidate_status { last_received { term: 2 index: 302 } } ignore_live_leader: false dest_uuid: "c324c9220fd14d3c817b18bbd3774439"
I20240503 02:09:14.388229 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.389787 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:14.390302 28583 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e12d823f5eb49f38d5fdc7ee2185bb4 in term 3.
I20240503 02:09:14.390488 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 442400993f8e43b79e429758d1f3c72f
I20240503 02:09:14.390630 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_33dbbeaec27e4fa6909562489deb85c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.390710 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33dbbeaec27e4fa6909562489deb85c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.390945 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f54f404aa16e495d994c597b7ea98869" } new_table_name: "default.Table_e30014f1b56649c080ec356295587b82" modify_external_catalogs: true
I20240503 02:09:14.391177 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.391254 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.391386 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 89440f04f9d349a187c296efab0816d2
I20240503 02:09:14.391441 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 89440f04f9d349a187c296efab0816d2 deleted and replaced by tablet 685ad701b1c143d4a77f6ec0c738c125
I20240503 02:09:14.391481 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_22e35ec738b34697afb7f6d08708a4b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.391549 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22e35ec738b34697afb7f6d08708a4b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.391675 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e43172b357ce45babdd7a4cfc2b8917a
I20240503 02:09:14.391726 27201 catalog_manager.cc:6554] ReplaceTablet: tablet e43172b357ce45babdd7a4cfc2b8917a deleted and replaced by tablet 6149f14895e34ea5994e3cbb94cf0bb4
I20240503 02:09:14.391767 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c839b3b37e0f40c487bdc964e6a6a999" } new_table_name: "default.Table_bda4d7a547de45e187f89fe8ff79deff" modify_external_catalogs: true
I20240503 02:09:14.391901 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9614b6572dfd443387ee3e4d676e8766
I20240503 02:09:14.391960 28638 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:14.391996 28638 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: success
I20240503 02:09:14.392014 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5ee8aa23d34f421f8f05b0982609667a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.392081 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ee8aa23d34f421f8f05b0982609667a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.392189 27203 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b004b105a74e408380a4fe15bacbff03
I20240503 02:09:14.392243 27203 catalog_manager.cc:6554] ReplaceTablet: tablet b004b105a74e408380a4fe15bacbff03 deleted and replaced by tablet 11c4e5ba1b274030b922c2234b0af7f3
I20240503 02:09:14.392295 27203 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_23855b5c24a34d70b99f876e28b2c299"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.392354 27203 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23855b5c24a34d70b99f876e28b2c299 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.393270 28638 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: acquiring CA information for follower catalog manager: success
I20240503 02:09:14.393651 28638 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:14.393958 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e12d823f5eb49f38d5fdc7ee2185bb4. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.394027 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:14.394058 28621 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.394091 28621 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:14.395054 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_90f372f788f64dd0af4c4b9718ade63a" } new_table_name: "default.Table_90367233df3b4c4680069a5fa47b4624" modify_external_catalogs: true
I20240503 02:09:14.395074 27199 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dc8099703c974c61a042d8d2c291a023" } new_table_name: "default.Table_ecb958f7920a44ff9aebead671143001" modify_external_catalogs: true
I20240503 02:09:14.396128 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e99d982dddd497988d2d7220b96aefa) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396243 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet d59c77e3e1444b2fb211363d59bd0631 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396317 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f with delete_type TABLET_DATA_DELETED (replaced by tablet 56b36a3207ff43c185993f0e9ea457fa) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396358 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 442400993f8e43b79e429758d1f3c72f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396394 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet 89440f04f9d349a187c296efab0816d2 with delete_type TABLET_DATA_DELETED (replaced by tablet 685ad701b1c143d4a77f6ec0c738c125) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396371 27561 tablet_service.cc:1514] Processing DeleteTablet for tablet 9614b6572dfd443387ee3e4d676e8766 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.396401 27563 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f with delete_type TABLET_DATA_DELETED (replaced by tablet 56b36a3207ff43c185993f0e9ea457fa) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.396430 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet e43172b357ce45babdd7a4cfc2b8917a with delete_type TABLET_DATA_DELETED (replaced by tablet 6149f14895e34ea5994e3cbb94cf0bb4) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396445 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet 442400993f8e43b79e429758d1f3c72f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.396466 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet 9614b6572dfd443387ee3e4d676e8766 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396517 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet b004b105a74e408380a4fe15bacbff03 with delete_type TABLET_DATA_DELETED (replaced by tablet 11c4e5ba1b274030b922c2234b0af7f3) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.396580 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99d982dddd497988d2d7220b96aefa (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.396621 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet e43172b357ce45babdd7a4cfc2b8917a with delete_type TABLET_DATA_DELETED (replaced by tablet 6149f14895e34ea5994e3cbb94cf0bb4) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.396659 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99d982dddd497988d2d7220b96aefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.396685 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet b004b105a74e408380a4fe15bacbff03 with delete_type TABLET_DATA_DELETED (replaced by tablet 11c4e5ba1b274030b922c2234b0af7f3) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.396767 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99d982dddd497988d2d7220b96aefa (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.396860 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99d982dddd497988d2d7220b96aefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.397858 27566 tablet_service.cc:1514] Processing DeleteTablet for tablet d59c77e3e1444b2fb211363d59bd0631 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.397924 27568 tablet_service.cc:1514] Processing DeleteTablet for tablet 89440f04f9d349a187c296efab0816d2 with delete_type TABLET_DATA_DELETED (replaced by tablet 685ad701b1c143d4a77f6ec0c738c125) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.398008 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0bfcd325883a414484aec17b7af9cb2f" } new_table_name: "default.Table_3954c32663f14e4b8cc0122d92f3c9cd" modify_external_catalogs: true
I20240503 02:09:14.396387 27564 tablet_service.cc:1514] Processing DeleteTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e99d982dddd497988d2d7220b96aefa) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.399132 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8918d4d6716f4974a4f660ca3eb42f17" } modify_external_catalogs: true
I20240503 02:09:14.399194 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8918d4d6716f4974a4f660ca3eb42f17" } modify_external_catalogs: true
I20240503 02:09:14.399137 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3925291fb0534d3582f1823d99624626" } modify_external_catalogs: true
I20240503 02:09:14.399521 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3925291fb0534d3582f1823d99624626" } modify_external_catalogs: true
I20240503 02:09:14.400032 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet dd3d3ca1b7ac478d8f5f942d6892d600 with delete_type TABLET_DATA_DELETED (replaced by tablet 2e99d982dddd497988d2d7220b96aefa) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400135 28649 mvcc.cc:205] Tried to move back new op lower bound from 7023420024366137344 to 7023420024305721344. Current Snapshot: MvccSnapshot[applied={T|T < 7023420024364392448}]
I20240503 02:09:14.400159 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet d59c77e3e1444b2fb211363d59bd0631 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400208 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ac26cca1be4b7e8cbfa227ab39e38f with delete_type TABLET_DATA_DELETED (replaced by tablet 56b36a3207ff43c185993f0e9ea457fa) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400251 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 442400993f8e43b79e429758d1f3c72f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400287 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 89440f04f9d349a187c296efab0816d2 with delete_type TABLET_DATA_DELETED (replaced by tablet 685ad701b1c143d4a77f6ec0c738c125) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400337 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet e43172b357ce45babdd7a4cfc2b8917a with delete_type TABLET_DATA_DELETED (replaced by tablet 6149f14895e34ea5994e3cbb94cf0bb4) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400388 27697 tablet_service.cc:1514] Processing DeleteTablet for tablet 9614b6572dfd443387ee3e4d676e8766 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400434 27696 tablet_service.cc:1514] Processing DeleteTablet for tablet b004b105a74e408380a4fe15bacbff03 with delete_type TABLET_DATA_DELETED (replaced by tablet 11c4e5ba1b274030b922c2234b0af7f3) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.400417 28655 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.400414 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:14.400471 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:14.400502 27695 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99d982dddd497988d2d7220b96aefa (DEFAULT_TABLE table=default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.400580 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99d982dddd497988d2d7220b96aefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.400210 28661 tablet_replica.cc:330] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.400725 28661 raft_consensus.cc:2229] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.400761 28661 raft_consensus.cc:2258] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.400846 28655 tablet_bootstrap.cc:654] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.400866 28661 ts_tablet_manager.cc:1875] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.402004 28655 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.402107 28656 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.402181 28655 ts_tablet_manager.cc:1367] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.402688 28655 raft_consensus.cc:348] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.403168 28655 raft_consensus.cc:374] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.403192 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 56b36a3207ff43c185993f0e9ea457fa (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.403249 28655 raft_consensus.cc:729] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.403290 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b36a3207ff43c185993f0e9ea457fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.403292 28655 consensus_queue.cc:260] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.403384 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 11a234271dc14a04a6feeb30e2fcd8f2 (DEFAULT_TABLE table=default.Table_32834d5c492a480090b2c41a0497a60f [id=cb1132c7b84948a6b0438c3f0bf5a53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.403441 28656 tablet_bootstrap.cc:654] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.403470 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a234271dc14a04a6feeb30e2fcd8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.403748 28653 tablet_replica.cc:330] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.403793 28653 raft_consensus.cc:2229] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.403820 28653 raft_consensus.cc:2258] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.403913 28653 ts_tablet_manager.cc:1875] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.404471 27430 tablet_service.cc:1467] Processing CreateTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 (DEFAULT_TABLE table=default.Table_df038b49f4354dc0ba3846a1f0abe3df [id=f8d3271eebfc495f8684be5e39d3fb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.404556 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31a2f92fdcf40928ee9e37d49a3ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.404942 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.405027 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6149f14895e34ea5994e3cbb94cf0bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.405565 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.405639 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6149f14895e34ea5994e3cbb94cf0bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.406090 27696 tablet_service.cc:1467] Processing CreateTablet for tablet a93b184492cf412d8e449e8c0527ea61 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.406170 27696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93b184492cf412d8e449e8c0527ea61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.406581 27428 tablet_service.cc:1467] Processing CreateTablet for tablet e7addc5958f04d1e83f77839e92201c3 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.406649 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7addc5958f04d1e83f77839e92201c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.407231 27703 tablet_service.cc:1467] Processing CreateTablet for tablet e7addc5958f04d1e83f77839e92201c3 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.407306 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7addc5958f04d1e83f77839e92201c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.407639 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 (DEFAULT_TABLE table=default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.407786 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685ad701b1c143d4a77f6ec0c738c125. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.407866 28654 tablet_replica.cc:330] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.407922 28654 raft_consensus.cc:2229] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.407955 28654 raft_consensus.cc:2258] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.408051 28654 ts_tablet_manager.cc:1875] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.408303 27698 tablet_service.cc:1467] Processing CreateTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 (DEFAULT_TABLE table=default.Table_df038b49f4354dc0ba3846a1f0abe3df [id=f8d3271eebfc495f8684be5e39d3fb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.408392 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31a2f92fdcf40928ee9e37d49a3ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.408732 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.408803 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c4e5ba1b274030b922c2234b0af7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.409289 28654 ts_tablet_manager.cc:1888] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.409350 28654 log.cc:1196] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9614b6572dfd443387ee3e4d676e8766
I20240503 02:09:14.409484 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 (DEFAULT_TABLE table=default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.409540 28654 ts_tablet_manager.cc:1909] T 9614b6572dfd443387ee3e4d676e8766 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.409562 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685ad701b1c143d4a77f6ec0c738c125. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.409797 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 56b36a3207ff43c185993f0e9ea457fa (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.409862 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b36a3207ff43c185993f0e9ea457fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.410198 28653 ts_tablet_manager.cc:1888] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.410235 28653 log.cc:1196] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/dd3d3ca1b7ac478d8f5f942d6892d600
I20240503 02:09:14.410419 28653 ts_tablet_manager.cc:1909] T dd3d3ca1b7ac478d8f5f942d6892d600 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.410642 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 11a234271dc14a04a6feeb30e2fcd8f2 (DEFAULT_TABLE table=default.Table_32834d5c492a480090b2c41a0497a60f [id=cb1132c7b84948a6b0438c3f0bf5a53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.410666 28653 tablet_replica.cc:330] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.410717 28653 raft_consensus.cc:2229] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.410723 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a234271dc14a04a6feeb30e2fcd8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.410744 28653 raft_consensus.cc:2258] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.407644 27500 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:14.410833 28653 ts_tablet_manager.cc:1875] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.411506 27433 tablet_service.cc:1467] Processing CreateTablet for tablet a93b184492cf412d8e449e8c0527ea61 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.411602 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93b184492cf412d8e449e8c0527ea61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.411625 28574 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" instance_seqno: 1714702146630581) as {username='slave'} at 127.26.117.1:41293; Asking this server to re-register.
I20240503 02:09:14.412022 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.412106 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c4e5ba1b274030b922c2234b0af7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.412634 28653 ts_tablet_manager.cc:1888] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.412676 28653 log.cc:1196] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d59c77e3e1444b2fb211363d59bd0631
I20240503 02:09:14.412855 28653 ts_tablet_manager.cc:1909] T d59c77e3e1444b2fb211363d59bd0631 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.412925 27500 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:14.413009 28654 tablet_replica.cc:330] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.413058 28654 raft_consensus.cc:2229] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.413125 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 39ffcd42bd284ef3ad4d3c49b010bdbd
I20240503 02:09:14.413305 28574 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.413384 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet dd3d3ca1b7ac478d8f5f942d6892d600 (table default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.413515 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e353654920a44994ae4889144db1b836
I20240503 02:09:14.413552 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.413712 28653 tablet_replica.cc:330] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.413765 28653 raft_consensus.cc:2229] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.413795 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet e353654920a44994ae4889144db1b836 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.413856 28653 raft_consensus.cc:2258] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.413947 28653 ts_tablet_manager.cc:1875] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.403395 28655 ts_tablet_manager.cc:1398] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.414305 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d59c77e3e1444b2fb211363d59bd0631 (table default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]) successfully deleted
I20240503 02:09:14.414561 28655 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.414566 27501 heartbeater.cc:503] Master 127.26.117.61:37847 was elected leader, sending a full tablet report...
I20240503 02:09:14.415218 28655 tablet_bootstrap.cc:654] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.415519 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_e30014f1b56649c080ec356295587b82 [id=b2a8365635134977a758514aa7a4351d]). Expected version 2 got 0
I20240503 02:09:14.415623 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644]). Expected version 1 got 0
I20240503 02:09:14.415621 28653 ts_tablet_manager.cc:1888] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.415669 28653 log.cc:1196] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/39ac26cca1be4b7e8cbfa227ab39e38f
I20240503 02:09:14.415675 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.415716 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a]). Expected version 1 got 0
I20240503 02:09:14.415748 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:14.415784 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 5443787e1d464baf93549b49571f5687 (table default.Table_3954c32663f14e4b8cc0122d92f3c9cd [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 1 got 0
I20240503 02:09:14.415817 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.415850 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.415855 28653 ts_tablet_manager.cc:1909] T 39ac26cca1be4b7e8cbfa227ab39e38f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.415953 28654 raft_consensus.cc:2258] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:14.415979 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 685ad701b1c143d4a77f6ec0c738c125 consensus state has no leader
W20240503 02:09:14.416047 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 56b36a3207ff43c185993f0e9ea457fa consensus state has no leader
I20240503 02:09:14.416066 28661 ts_tablet_manager.cc:1888] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.416074 28654 ts_tablet_manager.cc:1875] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.416106 28661 log.cc:1196] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/dd3d3ca1b7ac478d8f5f942d6892d600
I20240503 02:09:14.416281 28661 ts_tablet_manager.cc:1909] T dd3d3ca1b7ac478d8f5f942d6892d600 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.416462 28661 tablet_replica.cc:330] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.416529 28661 raft_consensus.cc:2229] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.416612 28661 raft_consensus.cc:2258] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.416646 28653 tablet_replica.cc:330] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.416688 28653 raft_consensus.cc:2229] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.416707 28661 ts_tablet_manager.cc:1875] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.416712 28653 raft_consensus.cc:2258] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.416790 28653 ts_tablet_manager.cc:1875] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.416970 27566 tablet_service.cc:1467] Processing CreateTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 (DEFAULT_TABLE table=default.Table_df038b49f4354dc0ba3846a1f0abe3df [id=f8d3271eebfc495f8684be5e39d3fb89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.417055 27566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31a2f92fdcf40928ee9e37d49a3ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.417789 28653 ts_tablet_manager.cc:1888] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.417829 28653 log.cc:1196] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/442400993f8e43b79e429758d1f3c72f
I20240503 02:09:14.417829 28661 ts_tablet_manager.cc:1888] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.417865 28661 log.cc:1196] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d59c77e3e1444b2fb211363d59bd0631
I20240503 02:09:14.418016 28653 ts_tablet_manager.cc:1909] T 442400993f8e43b79e429758d1f3c72f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.418027 28661 ts_tablet_manager.cc:1909] T d59c77e3e1444b2fb211363d59bd0631 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.418155 27561 tablet_service.cc:1467] Processing CreateTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 (DEFAULT_TABLE table=default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.418236 27561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685ad701b1c143d4a77f6ec0c738c125. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.419184 28654 ts_tablet_manager.cc:1888] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.419225 28654 log.cc:1196] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/39ac26cca1be4b7e8cbfa227ab39e38f
I20240503 02:09:14.419409 28654 ts_tablet_manager.cc:1909] T 39ac26cca1be4b7e8cbfa227ab39e38f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.419425 27559 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.419497 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet e353654920a44994ae4889144db1b836 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.419564 27557 tablet_service.cc:1467] Processing CreateTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda (DEFAULT_TABLE table=default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.419597 28653 tablet_replica.cc:330] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.419636 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b46bb7a647c64572bcae04d4dcdc1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.419641 28653 raft_consensus.cc:2229] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.419684 28653 raft_consensus.cc:2258] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.419649 28655 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.419729 28655 ts_tablet_manager.cc:1367] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:14.419764 28653 ts_tablet_manager.cc:1875] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.419893 28655 raft_consensus.cc:348] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.419943 28655 raft_consensus.cc:374] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.419960 28655 raft_consensus.cc:729] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.420011 28655 consensus_queue.cc:260] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.420133 28655 ts_tablet_manager.cc:1398] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.420296 28654 tablet_replica.cc:330] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.420346 28654 raft_consensus.cc:2229] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.420372 28654 raft_consensus.cc:2258] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.420454 28654 ts_tablet_manager.cc:1875] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.420176 28655 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.420714 27556 tablet_service.cc:1467] Processing CreateTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.420791 27556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a8e43c8d5554686aa9ecee00d265c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.421149 28655 tablet_bootstrap.cc:654] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.421578 28661 tablet_replica.cc:330] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.421630 28661 raft_consensus.cc:2229] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.421660 28661 raft_consensus.cc:2258] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.421662 28653 ts_tablet_manager.cc:1888] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.421700 28653 log.cc:1196] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/89440f04f9d349a187c296efab0816d2
I20240503 02:09:14.421746 28661 ts_tablet_manager.cc:1875] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.421872 28653 ts_tablet_manager.cc:1909] T 89440f04f9d349a187c296efab0816d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.421897 27555 tablet_service.cc:1467] Processing CreateTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 (DEFAULT_TABLE table=default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.421980 27555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00462246db1a4a53ba9f9a27a86ef538. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.422756 28664 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.422822 28653 tablet_replica.cc:330] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.422871 28653 raft_consensus.cc:2229] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.422914 28653 raft_consensus.cc:2258] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.423012 28664 tablet_bootstrap.cc:654] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.423012 28653 ts_tablet_manager.cc:1875] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.423031 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 (DEFAULT_TABLE table=default.Table_a58c505992a54fdfa6be0c230dd83adb [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.423220 27553 tablet_service.cc:1467] Processing CreateTablet for tablet 4dabf28ec9f14b5998910b185852b22c (DEFAULT_TABLE table=default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.423302 27552 tablet_service.cc:1467] Processing CreateTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.423362 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 (DEFAULT_TABLE table=default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.423446 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 (DEFAULT_TABLE table=default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.423506 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 56b36a3207ff43c185993f0e9ea457fa (DEFAULT_TABLE table=default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.424049 28653 ts_tablet_manager.cc:1888] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.424100 28653 log.cc:1196] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e43172b357ce45babdd7a4cfc2b8917a
I20240503 02:09:14.424269 28653 ts_tablet_manager.cc:1909] T e43172b357ce45babdd7a4cfc2b8917a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.424290 27565 tablet_service.cc:1467] Processing CreateTablet for tablet e7addc5958f04d1e83f77839e92201c3 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.424408 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7addc5958f04d1e83f77839e92201c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424566 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42df1b4d5b24fda8d1865c86f34d957. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424602 28653 tablet_replica.cc:330] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.424640 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc72a7c4997486280a2fdcb3bc07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424645 28653 raft_consensus.cc:2229] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.424674 28653 raft_consensus.cc:2258] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.424706 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56b36a3207ff43c185993f0e9ea457fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424764 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c4e5ba1b274030b922c2234b0af7f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424788 28653 ts_tablet_manager.cc:1875] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.424821 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6149f14895e34ea5994e3cbb94cf0bb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.424911 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dabf28ec9f14b5998910b185852b22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.425959 28653 ts_tablet_manager.cc:1888] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.426010 28653 log.cc:1196] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9614b6572dfd443387ee3e4d676e8766
I20240503 02:09:14.426103 27564 tablet_service.cc:1467] Processing CreateTablet for tablet a93b184492cf412d8e449e8c0527ea61 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.426177 28653 ts_tablet_manager.cc:1909] T 9614b6572dfd443387ee3e4d676e8766 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.426194 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93b184492cf412d8e449e8c0527ea61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.426733 28656 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.426771 28656 ts_tablet_manager.cc:1367] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240503 02:09:14.426908 28656 raft_consensus.cc:348] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.426959 28656 raft_consensus.cc:374] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.426977 28656 raft_consensus.cc:729] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.427014 28656 consensus_queue.cc:260] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.427093 28656 ts_tablet_manager.cc:1398] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.427170 28656 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.427470 28656 tablet_bootstrap.cc:654] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.428535 28661 ts_tablet_manager.cc:1888] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.428586 28661 log.cc:1196] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/39ac26cca1be4b7e8cbfa227ab39e38f
I20240503 02:09:14.428750 28661 ts_tablet_manager.cc:1909] T 39ac26cca1be4b7e8cbfa227ab39e38f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.428916 28661 tablet_replica.cc:330] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.428961 28661 raft_consensus.cc:2229] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.429073 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:14.429129 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 442400993f8e43b79e429758d1f3c72f (table default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]) successfully deleted
I20240503 02:09:14.429126 27433 tablet_service.cc:1467] Processing CreateTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda (DEFAULT_TABLE table=default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.429157 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
I20240503 02:09:14.429199 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b46bb7a647c64572bcae04d4dcdc1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.429198 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e43172b357ce45babdd7a4cfc2b8917a (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.429064 28661 raft_consensus.cc:2258] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.429301 28661 ts_tablet_manager.cc:1875] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.429409 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.429474 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a8e43c8d5554686aa9ecee00d265c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.429657 27702 tablet_service.cc:1467] Processing CreateTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.429742 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42df1b4d5b24fda8d1865c86f34d957. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.430382 28661 ts_tablet_manager.cc:1888] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.430423 28661 log.cc:1196] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/442400993f8e43b79e429758d1f3c72f
I20240503 02:09:14.430421 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 (DEFAULT_TABLE table=default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.430485 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00462246db1a4a53ba9f9a27a86ef538. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.430599 28661 ts_tablet_manager.cc:1909] T 442400993f8e43b79e429758d1f3c72f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.430934 28653 tablet_replica.cc:330] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.431001 28653 raft_consensus.cc:2229] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.431007 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet dd3d3ca1b7ac478d8f5f942d6892d600 (table default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.431047 28653 raft_consensus.cc:2258] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.431063 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d59c77e3e1444b2fb211363d59bd0631 (table default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]) successfully deleted
I20240503 02:09:14.431090 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:14.431133 28653 ts_tablet_manager.cc:1875] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.431432 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9614b6572dfd443387ee3e4d676e8766 (table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc]) successfully deleted
I20240503 02:09:14.431486 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 (DEFAULT_TABLE table=default.Table_a58c505992a54fdfa6be0c230dd83adb [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.431576 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc72a7c4997486280a2fdcb3bc07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.431741 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 39ac26cca1be4b7e8cbfa227ab39e38f (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]) successfully deleted
I20240503 02:09:14.432176 28653 ts_tablet_manager.cc:1888] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.432215 28653 log.cc:1196] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b004b105a74e408380a4fe15bacbff03
I20240503 02:09:14.432238 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 (DEFAULT_TABLE table=default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.432322 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00462246db1a4a53ba9f9a27a86ef538. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.432380 28653 ts_tablet_manager.cc:1909] T b004b105a74e408380a4fe15bacbff03 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.432644 27428 tablet_service.cc:1467] Processing CreateTablet for tablet 4dabf28ec9f14b5998910b185852b22c (DEFAULT_TABLE table=default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.432718 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dabf28ec9f14b5998910b185852b22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.433079 28661 tablet_replica.cc:330] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.433130 28661 raft_consensus.cc:2229] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.433156 28661 raft_consensus.cc:2258] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.433233 28661 ts_tablet_manager.cc:1875] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.433403 27424 tablet_service.cc:1467] Processing CreateTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.433444 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 442400993f8e43b79e429758d1f3c72f (table default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]) successfully deleted
I20240503 02:09:14.433477 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42df1b4d5b24fda8d1865c86f34d957. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.433717 27699 tablet_service.cc:1467] Processing CreateTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda (DEFAULT_TABLE table=default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.433800 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b46bb7a647c64572bcae04d4dcdc1cda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.434293 28653 tablet_replica.cc:330] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.434338 28653 raft_consensus.cc:2229] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.434331 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 4dabf28ec9f14b5998910b185852b22c (DEFAULT_TABLE table=default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.434384 28653 raft_consensus.cc:2258] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.434451 27696 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 (DEFAULT_TABLE table=default.Table_a58c505992a54fdfa6be0c230dd83adb [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.434468 28653 ts_tablet_manager.cc:1875] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.434520 27697 tablet_service.cc:1514] Processing DeleteTablet for tablet e353654920a44994ae4889144db1b836 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.434563 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 39ffcd42bd284ef3ad4d3c49b010bdbd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.434731 27636 heartbeater.cc:503] Master 127.26.117.61:37847 was elected leader, sending a full tablet report...
W20240503 02:09:14.435334 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet be7a66cc87834e739e958a2caf73de0c
W20240503 02:09:14.435371 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:14.435400 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 8127ac16597e40fa9b1590a61430aea0
W20240503 02:09:14.435428 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 38d89374090345758db303c33b460c9a
W20240503 02:09:14.435464 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:14.435501 28661 ts_tablet_manager.cc:1888] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.435513 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644]). Expected version 1 got 0
I20240503 02:09:14.435544 28661 log.cc:1196] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/89440f04f9d349a187c296efab0816d2
I20240503 02:09:14.435565 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_e30014f1b56649c080ec356295587b82 [id=b2a8365635134977a758514aa7a4351d]). Expected version 2 got 0
I20240503 02:09:14.435587 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.435622 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a]). Expected version 1 got 0
I20240503 02:09:14.435643 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:14.435668 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 5443787e1d464baf93549b49571f5687 (table default.Table_3954c32663f14e4b8cc0122d92f3c9cd [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 1 got 0
I20240503 02:09:14.435689 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.435712 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.435721 28661 ts_tablet_manager.cc:1909] T 89440f04f9d349a187c296efab0816d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.436304 28661 tablet_replica.cc:330] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.436311 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
I20240503 02:09:14.436354 28661 raft_consensus.cc:2229] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.436380 28661 raft_consensus.cc:2258] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.436424 27698 tablet_service.cc:1467] Processing CreateTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.436462 28661 ts_tablet_manager.cc:1875] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.436506 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a8e43c8d5554686aa9ecee00d265c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.436580 28653 ts_tablet_manager.cc:1888] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.436617 28653 log.cc:1196] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/39ffcd42bd284ef3ad4d3c49b010bdbd
I20240503 02:09:14.436784 28653 ts_tablet_manager.cc:1909] T 39ffcd42bd284ef3ad4d3c49b010bdbd P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.436945 28653 tablet_replica.cc:330] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.436987 28653 raft_consensus.cc:2229] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.437027 28653 raft_consensus.cc:2258] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.437110 28653 ts_tablet_manager.cc:1875] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.437773 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 11a234271dc14a04a6feeb30e2fcd8f2 (DEFAULT_TABLE table=default.Table_32834d5c492a480090b2c41a0497a60f [id=cb1132c7b84948a6b0438c3f0bf5a53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.437858 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a234271dc14a04a6feeb30e2fcd8f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.438769 28661 ts_tablet_manager.cc:1888] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.438813 28661 log.cc:1196] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e43172b357ce45babdd7a4cfc2b8917a
I20240503 02:09:14.438989 28661 ts_tablet_manager.cc:1909] T e43172b357ce45babdd7a4cfc2b8917a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.439451 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dabf28ec9f14b5998910b185852b22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.439674 27696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bc72a7c4997486280a2fdcb3bc07c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.440027 28653 ts_tablet_manager.cc:1888] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.440068 28653 log.cc:1196] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e353654920a44994ae4889144db1b836
I20240503 02:09:14.440234 28653 ts_tablet_manager.cc:1909] T e353654920a44994ae4889144db1b836 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.424413 28654 ts_tablet_manager.cc:1888] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.441031 28654 log.cc:1196] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/442400993f8e43b79e429758d1f3c72f
I20240503 02:09:14.441289 28698 tablet.cc:1722] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.441517 28654 ts_tablet_manager.cc:1909] T 442400993f8e43b79e429758d1f3c72f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.441857 28654 tablet_replica.cc:330] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.441911 28654 raft_consensus.cc:2229] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.441941 28654 raft_consensus.cc:2258] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.442026 28654 ts_tablet_manager.cc:1875] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.442270 28698 tablet.cc:1722] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.443222 28654 ts_tablet_manager.cc:1888] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20240503 02:09:14.444561 27190 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 56b36a3207ff43c185993f0e9ea457fa consensus state has no leader
I20240503 02:09:14.445402 28661 tablet_replica.cc:330] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.445480 28661 raft_consensus.cc:2229] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.445569 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e43172b357ce45babdd7a4cfc2b8917a (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.445612 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9614b6572dfd443387ee3e4d676e8766 (table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc]) successfully deleted
I20240503 02:09:14.445658 28654 log.cc:1196] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e43172b357ce45babdd7a4cfc2b8917a
I20240503 02:09:14.445686 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b004b105a74e408380a4fe15bacbff03 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.445726 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (table default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]) successfully deleted
I20240503 02:09:14.445765 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e353654920a44994ae4889144db1b836 (table default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]) successfully deleted
I20240503 02:09:14.445855 28654 ts_tablet_manager.cc:1909] T e43172b357ce45babdd7a4cfc2b8917a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.445937 28661 raft_consensus.cc:2258] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.446064 28661 ts_tablet_manager.cc:1875] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.446686 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 442400993f8e43b79e429758d1f3c72f (table default.Table_16a939c3436b4c0fabceb3838811d215 [id=4caff2ed23254f40a91a55aa7e543338]) successfully deleted
I20240503 02:09:14.446801 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a] alter complete (version 1)
I20240503 02:09:14.447135 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:14.447409 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e43172b357ce45babdd7a4cfc2b8917a (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.443809 28698 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.447813 28703 tablet.cc:1722] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.448670 28574 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" instance_seqno: 1714702146750084) as {username='slave'} at 127.26.117.2:40303; Asking this server to re-register.
I20240503 02:09:14.448856 27635 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:14.449386 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644] alter complete (version 1)
I20240503 02:09:14.449474 28574 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.449807 28655 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.449896 28655 ts_tablet_manager.cc:1367] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.001s
I20240503 02:09:14.450245 28705 tablet.cc:1722] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.450592 28706 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.450664 28654 tablet_replica.cc:330] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.451472 28654 raft_consensus.cc:2229] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.451540 28698 tablet.cc:1722] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.451562 28654 raft_consensus.cc:2258] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.451799 28654 ts_tablet_manager.cc:1875] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.451977 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_e30014f1b56649c080ec356295587b82 [id=b2a8365635134977a758514aa7a4351d] alter complete (version 2)
I20240503 02:09:14.450081 28655 raft_consensus.cc:348] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.453246 28655 raft_consensus.cc:374] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.453310 28655 raft_consensus.cc:729] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.453441 28688 tablet.cc:1722] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.453473 28655 consensus_queue.cc:260] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.453881 28655 ts_tablet_manager.cc:1398] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:14.454066 28655 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.454625 28655 tablet_bootstrap.cc:654] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.455116 28664 tablet_bootstrap.cc:492] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.455217 28664 ts_tablet_manager.cc:1367] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240503 02:09:14.455405 28664 raft_consensus.cc:348] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.455526 28664 raft_consensus.cc:374] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.455617 27771 heartbeater.cc:503] Master 127.26.117.61:37847 was elected leader, sending a full tablet report...
I20240503 02:09:14.455637 28664 raft_consensus.cc:729] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.455873 28664 consensus_queue.cc:260] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.456045 28656 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.456085 28656 ts_tablet_manager.cc:1367] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240503 02:09:14.456313 28664 ts_tablet_manager.cc:1398] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.456315 28656 raft_consensus.cc:348] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.456377 28656 raft_consensus.cc:374] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.456399 28656 raft_consensus.cc:729] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.456435 28656 consensus_queue.cc:260] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.456528 28656 ts_tablet_manager.cc:1398] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:14.456539 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.456578 28656 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.456605 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6b1e79e771df40f7abcdecaf683ae61b (table default.Table_e30014f1b56649c080ec356295587b82 [id=b2a8365635134977a758514aa7a4351d]). Expected version 2 got 0
I20240503 02:09:14.456655 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]). Expected version 1 got 0
I20240503 02:09:14.456466 28664 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.456697 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.456723 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.456880 28656 tablet_bootstrap.cc:654] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.457139 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b] alter complete (version 1)
I20240503 02:09:14.457690 28710 tablet.cc:1722] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.458570 27770 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:14.458696 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3954c32663f14e4b8cc0122d92f3c9cd [id=c46a0a1c65dd47e38382f2b2d19bff05] alter complete (version 1)
I20240503 02:09:14.459080 28710 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.459956 28664 tablet_bootstrap.cc:654] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.460172 28710 tablet.cc:1722] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.455751 28688 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.460723 28574 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" instance_seqno: 1714702146863903) as {username='slave'} at 127.26.117.3:48645; Asking this server to re-register.
I20240503 02:09:14.460892 27770 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:14.461228 28710 tablet.cc:1722] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.461128 28705 tablet.cc:1722] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.461925 28574 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.463392 28661 ts_tablet_manager.cc:1888] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.463475 28661 log.cc:1196] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9614b6572dfd443387ee3e4d676e8766
I20240503 02:09:14.463696 28661 ts_tablet_manager.cc:1909] T 9614b6572dfd443387ee3e4d676e8766 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.463948 28661 tablet_replica.cc:330] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.463989 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9614b6572dfd443387ee3e4d676e8766 (table default.Table_29c62440de5041adb7dc3747d41924ee [id=93d01889ff5b454e86796788b3862fdc]) successfully deleted
I20240503 02:09:14.464023 28661 raft_consensus.cc:2229] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.464085 28661 raft_consensus.cc:2258] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.464460 28686 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.464898 28661 ts_tablet_manager.cc:1875] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.466130 28654 ts_tablet_manager.cc:1888] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.466218 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4] alter complete (version 1)
I20240503 02:09:14.466259 28654 log.cc:1196] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b004b105a74e408380a4fe15bacbff03
I20240503 02:09:14.466688 28706 tablet.cc:1722] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.467459 28661 ts_tablet_manager.cc:1888] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.467528 28654 ts_tablet_manager.cc:1909] T b004b105a74e408380a4fe15bacbff03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.467561 28661 log.cc:1196] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b004b105a74e408380a4fe15bacbff03
I20240503 02:09:14.467792 28661 ts_tablet_manager.cc:1909] T b004b105a74e408380a4fe15bacbff03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.467883 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b004b105a74e408380a4fe15bacbff03 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.468008 28654 tablet_replica.cc:330] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.468066 28661 tablet_replica.cc:330] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.468073 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b004b105a74e408380a4fe15bacbff03 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.468111 28661 raft_consensus.cc:2229] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.468138 28661 raft_consensus.cc:2258] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.468190 28686 tablet.cc:1722] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.468286 28661 ts_tablet_manager.cc:1875] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.468348 28654 raft_consensus.cc:2229] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.468616 28654 raft_consensus.cc:2258] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.468775 28698 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.468839 28654 ts_tablet_manager.cc:1875] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.469368 28661 ts_tablet_manager.cc:1888] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.469473 28661 log.cc:1196] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e353654920a44994ae4889144db1b836
I20240503 02:09:14.469699 28661 ts_tablet_manager.cc:1909] T e353654920a44994ae4889144db1b836 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.469460 28655 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.469975 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e353654920a44994ae4889144db1b836 (table default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]) successfully deleted
I20240503 02:09:14.470007 28655 ts_tablet_manager.cc:1367] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240503 02:09:14.469974 28661 tablet_replica.cc:330] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.470130 28661 raft_consensus.cc:2229] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.470149 28655 raft_consensus.cc:348] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.470209 28655 raft_consensus.cc:374] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.470230 28655 raft_consensus.cc:729] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.470196 28661 raft_consensus.cc:2258] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.470273 28655 consensus_queue.cc:260] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.470347 28655 ts_tablet_manager.cc:1398] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.470463 28661 ts_tablet_manager.cc:1875] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.471498 28661 ts_tablet_manager.cc:1888] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.471596 28661 log.cc:1196] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/39ffcd42bd284ef3ad4d3c49b010bdbd
I20240503 02:09:14.471817 28661 ts_tablet_manager.cc:1909] T 39ffcd42bd284ef3ad4d3c49b010bdbd P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.472095 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (table default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]) successfully deleted
W20240503 02:09:14.472380 27523 consensus_peers.cc:485] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Couldn't send request to peer c2be543b8aad4e80a58d6b4ab669faae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89440f04f9d349a187c296efab0816d2. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:14.472442 28656 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.472489 28656 ts_tablet_manager.cc:1367] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:14.471937 28655 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.472610 28656 raft_consensus.cc:348] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.472659 28656 raft_consensus.cc:374] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.472677 28656 raft_consensus.cc:729] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.472712 28656 consensus_queue.cc:260] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.472788 28656 ts_tablet_manager.cc:1398] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.472848 28656 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.472873 28655 tablet_bootstrap.cc:654] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.473177 28656 tablet_bootstrap.cc:654] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.474025 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.474305 28656 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.474347 28656 ts_tablet_manager.cc:1367] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.474479 28656 raft_consensus.cc:348] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.474539 28656 raft_consensus.cc:374] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.474556 28656 raft_consensus.cc:729] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.474591 28656 consensus_queue.cc:260] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.474658 28656 ts_tablet_manager.cc:1398] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.474718 28656 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.475028 28656 tablet_bootstrap.cc:654] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.475029 28664 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.475068 28664 ts_tablet_manager.cc:1367] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240503 02:09:14.475345 28664 raft_consensus.cc:348] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.475435 28664 raft_consensus.cc:374] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.475507 28664 raft_consensus.cc:729] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.476044 28656 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.476087 28656 ts_tablet_manager.cc:1367] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.476197 28656 raft_consensus.cc:348] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.476264 28656 raft_consensus.cc:374] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.476282 28656 raft_consensus.cc:729] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.476316 28656 consensus_queue.cc:260] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.476393 28656 ts_tablet_manager.cc:1398] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.476438 28656 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.476848 28656 tablet_bootstrap.cc:654] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.477056 28664 consensus_queue.cc:260] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.477155 28664 ts_tablet_manager.cc:1398] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.477262 28664 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.477440 28656 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.477484 28656 ts_tablet_manager.cc:1367] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.477607 28656 raft_consensus.cc:348] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.477643 28664 tablet_bootstrap.cc:654] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.477655 28656 raft_consensus.cc:374] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.477674 28656 raft_consensus.cc:729] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.477710 28656 consensus_queue.cc:260] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.477779 28656 ts_tablet_manager.cc:1398] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.477821 28656 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.478112 28656 tablet_bootstrap.cc:654] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.474484 28654 ts_tablet_manager.cc:1888] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.479010 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.479440 28656 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.479645 28656 ts_tablet_manager.cc:1367] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.479777 28656 raft_consensus.cc:348] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.479828 28656 raft_consensus.cc:374] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.479846 28656 raft_consensus.cc:729] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.479879 28656 consensus_queue.cc:260] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.479959 28656 ts_tablet_manager.cc:1398] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.480008 28656 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.480296 28656 tablet_bootstrap.cc:654] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.481046 28654 log.cc:1196] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d59c77e3e1444b2fb211363d59bd0631
I20240503 02:09:14.481259 28654 ts_tablet_manager.cc:1909] T d59c77e3e1444b2fb211363d59bd0631 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.481359 28656 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.481400 28656 ts_tablet_manager.cc:1367] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.481482 28654 tablet_replica.cc:330] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.481525 28654 raft_consensus.cc:2229] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.481532 28656 raft_consensus.cc:348] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.481554 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d59c77e3e1444b2fb211363d59bd0631 (table default.Table_b795b4876be143edb345a11ccda4a55b [id=9b85049559f44f7680639644689ea83a]) successfully deleted
I20240503 02:09:14.481576 28656 raft_consensus.cc:374] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.481614 28656 raft_consensus.cc:729] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.481644 28656 consensus_queue.cc:260] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.481724 28656 ts_tablet_manager.cc:1398] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.481814 28656 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.482128 28656 tablet_bootstrap.cc:654] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.482662 28656 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.482703 28656 ts_tablet_manager.cc:1367] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.482831 28656 raft_consensus.cc:348] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.482877 28656 raft_consensus.cc:374] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.482893 28656 raft_consensus.cc:729] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.482933 28656 consensus_queue.cc:260] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.483002 28656 ts_tablet_manager.cc:1398] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.483044 28656 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.483353 28656 tablet_bootstrap.cc:654] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.481575 28654 raft_consensus.cc:2258] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.484387 28654 ts_tablet_manager.cc:1875] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.485589 28656 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.485642 28656 ts_tablet_manager.cc:1367] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.485639 28655 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.485682 28655 ts_tablet_manager.cc:1367] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240503 02:09:14.485790 28664 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.485769 28656 raft_consensus.cc:348] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.485819 28656 raft_consensus.cc:374] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.485838 28656 raft_consensus.cc:729] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.485823 28655 raft_consensus.cc:348] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.485875 28655 raft_consensus.cc:374] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.485893 28655 raft_consensus.cc:729] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.485898 28656 consensus_queue.cc:260] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.485926 28655 consensus_queue.cc:260] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.485991 28656 ts_tablet_manager.cc:1398] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.486006 28655 ts_tablet_manager.cc:1398] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.486068 28655 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.486142 28664 ts_tablet_manager.cc:1367] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:14.486348 28656 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.486407 28655 tablet_bootstrap.cc:654] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.486454 28664 raft_consensus.cc:348] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.486550 28664 raft_consensus.cc:374] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.486572 28664 raft_consensus.cc:729] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.486621 28664 consensus_queue.cc:260] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.486694 28656 tablet_bootstrap.cc:654] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.486701 28664 ts_tablet_manager.cc:1398] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.487072 27848 raft_consensus.cc:482] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.487118 27848 raft_consensus.cc:504] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.487206 27848 leader_election.cc:288] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.487351 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.487454 28654 ts_tablet_manager.cc:1888] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.487674 28654 log.cc:1196] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/89440f04f9d349a187c296efab0816d2
W20240503 02:09:14.487761 27524 leader_election.cc:341] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:14.487922 28654 ts_tablet_manager.cc:1909] T 89440f04f9d349a187c296efab0816d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.488055 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.488258 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 89440f04f9d349a187c296efab0816d2 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
W20240503 02:09:14.488319 27523 leader_election.cc:341] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:14.488360 27523 leader_election.cc:302] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:14.488425 27848 raft_consensus.cc:2726] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:14.488569 28654 tablet_replica.cc:330] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.488684 28654 raft_consensus.cc:2229] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.488790 28654 raft_consensus.cc:2258] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.488930 28654 ts_tablet_manager.cc:1875] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.489027 28664 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.489758 28664 tablet_bootstrap.cc:654] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.490131 28654 ts_tablet_manager.cc:1888] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.490187 28654 log.cc:1196] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/dd3d3ca1b7ac478d8f5f942d6892d600
I20240503 02:09:14.490366 28654 ts_tablet_manager.cc:1909] T dd3d3ca1b7ac478d8f5f942d6892d600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.490413 28655 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.490460 28655 ts_tablet_manager.cc:1367] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:14.490592 28654 tablet_replica.cc:330] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.490579 28655 raft_consensus.cc:348] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.490630 28655 raft_consensus.cc:374] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.490633 28654 raft_consensus.cc:2229] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.490649 28655 raft_consensus.cc:729] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.490660 28654 raft_consensus.cc:2258] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.490671 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet dd3d3ca1b7ac478d8f5f942d6892d600 (table default.Table_7a4dd759006b4f999a36751c7b62ae48 [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.490684 28655 consensus_queue.cc:260] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.490764 28655 ts_tablet_manager.cc:1398] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.490811 28655 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.490747 28654 ts_tablet_manager.cc:1875] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.491336 28655 tablet_bootstrap.cc:654] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.491849 28664 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.491889 28664 ts_tablet_manager.cc:1367] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.492017 28664 raft_consensus.cc:348] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.492069 28664 raft_consensus.cc:374] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.492085 28664 raft_consensus.cc:729] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.492102 28654 ts_tablet_manager.cc:1888] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.492121 28664 consensus_queue.cc:260] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.492146 28654 log.cc:1196] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/39ffcd42bd284ef3ad4d3c49b010bdbd
I20240503 02:09:14.492194 28664 ts_tablet_manager.cc:1398] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.492254 28664 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.492290 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.492319 28654 ts_tablet_manager.cc:1909] T 39ffcd42bd284ef3ad4d3c49b010bdbd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.492503 28654 tablet_replica.cc:330] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.492549 28654 raft_consensus.cc:2229] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.492563 28664 tablet_bootstrap.cc:654] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.492581 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 39ffcd42bd284ef3ad4d3c49b010bdbd (table default.Table_8918d4d6716f4974a4f660ca3eb42f17 [id=918166d10cee4f07a6e5edcc4e437e42]) successfully deleted
I20240503 02:09:14.492591 28654 raft_consensus.cc:2258] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.492672 28654 ts_tablet_manager.cc:1875] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.493031 28664 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.493069 28664 ts_tablet_manager.cc:1367] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.493191 28664 raft_consensus.cc:348] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.493243 28664 raft_consensus.cc:374] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.493258 28664 raft_consensus.cc:729] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.493290 28664 consensus_queue.cc:260] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.493388 28664 ts_tablet_manager.cc:1398] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.493436 28664 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.493726 28654 ts_tablet_manager.cc:1888] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.493768 28664 tablet_bootstrap.cc:654] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.493808 28654 log.cc:1196] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e353654920a44994ae4889144db1b836
I20240503 02:09:14.493988 28654 ts_tablet_manager.cc:1909] T e353654920a44994ae4889144db1b836 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.494220 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e353654920a44994ae4889144db1b836 (table default.Table_3925291fb0534d3582f1823d99624626 [id=331e07089a6b42d99d4526e1cf90e391]) successfully deleted
I20240503 02:09:14.494443 28655 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.494478 28655 ts_tablet_manager.cc:1367] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:14.494599 28655 raft_consensus.cc:348] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.494648 28655 raft_consensus.cc:374] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.494665 28655 raft_consensus.cc:729] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.494709 28655 consensus_queue.cc:260] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.494782 28655 ts_tablet_manager.cc:1398] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.494840 28655 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.494982 28664 tablet_bootstrap.cc:492] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.495031 28664 ts_tablet_manager.cc:1367] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.495143 28655 tablet_bootstrap.cc:654] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.495158 28664 raft_consensus.cc:348] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.495209 28664 raft_consensus.cc:374] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.495227 28664 raft_consensus.cc:729] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.495260 28664 consensus_queue.cc:260] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.495332 28664 ts_tablet_manager.cc:1398] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.495503 28656 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.495539 28656 ts_tablet_manager.cc:1367] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:14.495647 28656 raft_consensus.cc:348] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.495699 28656 raft_consensus.cc:374] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.495715 28656 raft_consensus.cc:729] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.495744 28656 consensus_queue.cc:260] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.495816 28656 ts_tablet_manager.cc:1398] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.495860 28656 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.496168 28656 tablet_bootstrap.cc:654] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.496616 28664 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.496613 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.496623 28656 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.496660 28656 ts_tablet_manager.cc:1367] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.496783 28656 raft_consensus.cc:348] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.496840 28656 raft_consensus.cc:374] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.496858 28656 raft_consensus.cc:729] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.496894 28656 consensus_queue.cc:260] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.496964 28656 ts_tablet_manager.cc:1398] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.497016 28656 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.497397 28664 tablet_bootstrap.cc:654] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.497694 28656 tablet_bootstrap.cc:654] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.498392 28655 tablet_bootstrap.cc:492] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.498437 28655 ts_tablet_manager.cc:1367] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:14.498553 28664 tablet_bootstrap.cc:492] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.498589 28664 ts_tablet_manager.cc:1367] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.498626 28655 raft_consensus.cc:348] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.498672 28655 raft_consensus.cc:374] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.498687 28655 raft_consensus.cc:729] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.498682 28664 raft_consensus.cc:348] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.498716 28664 raft_consensus.cc:374] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.498732 28664 raft_consensus.cc:729] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.498726 28655 consensus_queue.cc:260] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.498764 28664 consensus_queue.cc:260] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.498798 28655 ts_tablet_manager.cc:1398] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.498832 28664 ts_tablet_manager.cc:1398] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.498843 28655 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.498888 28664 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.499153 28655 tablet_bootstrap.cc:654] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.499218 28664 tablet_bootstrap.cc:654] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.499855 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.499889 28656 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.499926 28656 ts_tablet_manager.cc:1367] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:14.500036 28656 raft_consensus.cc:348] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.500092 28656 raft_consensus.cc:374] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.500110 28656 raft_consensus.cc:729] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.500150 28656 consensus_queue.cc:260] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.500223 28656 ts_tablet_manager.cc:1398] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.500300 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.500327 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.500917 28655 tablet_bootstrap.cc:492] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.500921 28664 tablet_bootstrap.cc:492] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.500960 28664 ts_tablet_manager.cc:1367] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.500962 28655 ts_tablet_manager.cc:1367] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.501081 28664 raft_consensus.cc:348] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.501098 28655 raft_consensus.cc:348] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.501130 28664 raft_consensus.cc:374] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.501147 28655 raft_consensus.cc:374] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.501148 28664 raft_consensus.cc:729] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.501165 28655 raft_consensus.cc:729] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.501183 28664 consensus_queue.cc:260] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.501204 28655 consensus_queue.cc:260] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.501252 28664 ts_tablet_manager.cc:1398] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.501277 28655 ts_tablet_manager.cc:1398] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.501304 28664 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.501648 28664 tablet_bootstrap.cc:654] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.502033 28655 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.502357 28655 tablet_bootstrap.cc:654] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.502368 28656 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.502700 28656 tablet_bootstrap.cc:654] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.503224 28656 tablet_bootstrap.cc:492] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.503253 28664 tablet_bootstrap.cc:492] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.503265 28656 ts_tablet_manager.cc:1367] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.503286 28693 raft_consensus.cc:482] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.503331 28693 raft_consensus.cc:504] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.503403 28656 raft_consensus.cc:348] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.503293 28664 ts_tablet_manager.cc:1367] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.503454 28656 raft_consensus.cc:374] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.503473 28656 raft_consensus.cc:729] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.503507 28656 consensus_queue.cc:260] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.503579 28656 ts_tablet_manager.cc:1398] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.503568 28664 raft_consensus.cc:348] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.503621 28664 raft_consensus.cc:374] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.503604 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99d982dddd497988d2d7220b96aefa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.503638 28664 raft_consensus.cc:729] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.503649 28656 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.503654 27449 raft_consensus.cc:2454] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.503672 28664 consensus_queue.cc:260] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.503742 28664 ts_tablet_manager.cc:1398] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.503765 27659 leader_election.cc:302] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.503794 28664 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.503441 28693 leader_election.cc:288] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.503959 28693 raft_consensus.cc:2781] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.503958 28656 tablet_bootstrap.cc:654] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.503981 28693 raft_consensus.cc:482] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.504000 28693 raft_consensus.cc:3037] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.504096 28664 tablet_bootstrap.cc:654] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.504619 28693 raft_consensus.cc:504] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.504729 28693 leader_election.cc:288] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.504813 28655 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.504853 28655 ts_tablet_manager.cc:1367] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.504972 28655 raft_consensus.cc:348] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.505040 28655 raft_consensus.cc:374] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.505065 28655 raft_consensus.cc:729] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.505100 28655 consensus_queue.cc:260] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.505195 28655 ts_tablet_manager.cc:1398] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.505246 28655 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.505426 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99d982dddd497988d2d7220b96aefa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.505506 27587 raft_consensus.cc:2454] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.505597 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99d982dddd497988d2d7220b96aefa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.505641 27585 raft_consensus.cc:3037] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.506186 27585 raft_consensus.cc:2454] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.506354 27656 leader_election.cc:302] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.506616 28655 tablet_bootstrap.cc:654] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.506796 28693 raft_consensus.cc:2781] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.506841 28693 raft_consensus.cc:686] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.506875 28693 consensus_queue.cc:237] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.507164 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99d982dddd497988d2d7220b96aefa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.507218 27449 raft_consensus.cc:3037] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.507789 28664 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.507807 28656 tablet_bootstrap.cc:492] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.507827 28664 ts_tablet_manager.cc:1367] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:14.507846 28656 ts_tablet_manager.cc:1367] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:14.507856 27449 raft_consensus.cc:2454] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.507941 28664 raft_consensus.cc:348] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.507956 28656 raft_consensus.cc:348] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.507988 28664 raft_consensus.cc:374] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.508003 28656 raft_consensus.cc:374] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.508021 28664 raft_consensus.cc:729] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.508023 28656 raft_consensus.cc:729] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.508054 28664 consensus_queue.cc:260] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.508061 28656 consensus_queue.cc:260] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.508131 28656 ts_tablet_manager.cc:1398] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.508148 28664 ts_tablet_manager.cc:1398] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.508194 28664 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.508491 28664 tablet_bootstrap.cc:654] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.509116 27198 catalog_manager.cc:5484] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.509706 28655 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.509743 28655 ts_tablet_manager.cc:1367] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:14.509860 28655 raft_consensus.cc:348] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.510022 28655 raft_consensus.cc:374] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.510044 28655 raft_consensus.cc:729] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.510080 28655 consensus_queue.cc:260] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.510802 28655 ts_tablet_manager.cc:1398] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.511312 28655 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.511768 28655 tablet_bootstrap.cc:654] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.512492 28664 tablet_bootstrap.cc:492] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.512532 28655 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.512543 28664 ts_tablet_manager.cc:1367] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:14.512570 28655 ts_tablet_manager.cc:1367] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.512678 28664 raft_consensus.cc:348] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.512750 28664 raft_consensus.cc:374] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.512722 28655 raft_consensus.cc:348] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.512770 28664 raft_consensus.cc:729] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.512773 28655 raft_consensus.cc:374] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.512794 28655 raft_consensus.cc:729] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.512814 28664 consensus_queue.cc:260] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.512845 28655 consensus_queue.cc:260] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.512892 28664 ts_tablet_manager.cc:1398] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.512966 28664 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.513283 28664 tablet_bootstrap.cc:654] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.514425 28655 ts_tablet_manager.cc:1398] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.514513 28655 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.515026 28655 tablet_bootstrap.cc:654] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.515519 28655 tablet_bootstrap.cc:492] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.515556 28655 ts_tablet_manager.cc:1367] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.515647 28655 raft_consensus.cc:348] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.515689 28655 raft_consensus.cc:374] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.515700 28655 raft_consensus.cc:729] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.515727 28655 consensus_queue.cc:260] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.515805 28655 ts_tablet_manager.cc:1398] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.515861 28655 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.516161 28655 tablet_bootstrap.cc:654] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.516826 28655 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.516868 28655 ts_tablet_manager.cc:1367] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.517000 28655 raft_consensus.cc:348] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.517056 28655 raft_consensus.cc:374] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.517091 28655 raft_consensus.cc:729] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.517123 28655 consensus_queue.cc:260] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.517211 28655 ts_tablet_manager.cc:1398] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.517258 28655 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.517588 28655 tablet_bootstrap.cc:654] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.518810 28655 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.518857 28655 ts_tablet_manager.cc:1367] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.518956 28655 raft_consensus.cc:348] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.518998 28655 raft_consensus.cc:374] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.519009 28655 raft_consensus.cc:729] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.519037 28655 consensus_queue.cc:260] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.519112 28655 ts_tablet_manager.cc:1398] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.519307 28664 tablet_bootstrap.cc:492] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.519347 28664 ts_tablet_manager.cc:1367] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:14.519555 28664 raft_consensus.cc:348] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.519593 28664 raft_consensus.cc:374] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.519604 28664 raft_consensus.cc:729] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.519620 28664 consensus_queue.cc:260] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.519688 28664 ts_tablet_manager.cc:1398] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.519730 28664 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.520020 28664 tablet_bootstrap.cc:654] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.520931 28664 tablet_bootstrap.cc:492] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.520973 28664 ts_tablet_manager.cc:1367] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.521091 28664 raft_consensus.cc:348] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.521158 28664 raft_consensus.cc:374] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.521176 28664 raft_consensus.cc:729] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.521216 28664 consensus_queue.cc:260] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.521291 28664 ts_tablet_manager.cc:1398] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.521580 27848 raft_consensus.cc:482] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.521618 28664 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.521629 27848 raft_consensus.cc:504] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.521718 27848 leader_election.cc:288] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.521888 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42df1b4d5b24fda8d1865c86f34d957" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.521941 27449 raft_consensus.cc:2454] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.522042 27524 leader_election.cc:302] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.522117 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42df1b4d5b24fda8d1865c86f34d957" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.522173 27721 raft_consensus.cc:2454] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.521929 28664 tablet_bootstrap.cc:654] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.522199 27848 raft_consensus.cc:2781] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.522228 27848 raft_consensus.cc:482] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.522251 27848 raft_consensus.cc:3037] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.522756 28664 tablet_bootstrap.cc:492] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.522765 27848 raft_consensus.cc:504] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.522805 28664 ts_tablet_manager.cc:1367] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.522871 27848 leader_election.cc:288] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.522914 28664 raft_consensus.cc:348] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.523118 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42df1b4d5b24fda8d1865c86f34d957" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.523172 27449 raft_consensus.cc:3037] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.523088 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42df1b4d5b24fda8d1865c86f34d957" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.523243 27721 raft_consensus.cc:3037] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.523612 27449 raft_consensus.cc:2454] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.523849 27524 leader_election.cc:302] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.523875 27721 raft_consensus.cc:2454] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.523927 27848 raft_consensus.cc:2781] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.523967 27848 raft_consensus.cc:686] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.524014 27848 consensus_queue.cc:237] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.523229 28664 raft_consensus.cc:374] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.524158 28664 raft_consensus.cc:729] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.524255 28664 consensus_queue.cc:260] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.524698 27198 catalog_manager.cc:5484] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:14.524567 28664 ts_tablet_manager.cc:1398] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.525025 28664 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.525653 28664 tablet_bootstrap.cc:654] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.529598 28664 tablet_bootstrap.cc:492] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.529639 28664 ts_tablet_manager.cc:1367] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:14.529749 28664 raft_consensus.cc:348] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.529793 28664 raft_consensus.cc:374] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.529809 28664 raft_consensus.cc:729] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.529843 28664 consensus_queue.cc:260] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.529917 28664 ts_tablet_manager.cc:1398] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.534188 28740 raft_consensus.cc:482] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.534250 28740 raft_consensus.cc:504] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.534368 28740 leader_election.cc:288] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.534471 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31a2f92fdcf40928ee9e37d49a3ac47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.534502 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31a2f92fdcf40928ee9e37d49a3ac47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.534552 27449 raft_consensus.cc:2454] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.534552 27585 raft_consensus.cc:2454] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.534657 27656 leader_election.cc:302] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.534730 28740 raft_consensus.cc:2781] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.534781 28740 raft_consensus.cc:482] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.534807 28740 raft_consensus.cc:3037] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.535619 28740 raft_consensus.cc:504] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.535748 28740 leader_election.cc:288] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.535915 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31a2f92fdcf40928ee9e37d49a3ac47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.535914 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31a2f92fdcf40928ee9e37d49a3ac47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.535974 27585 raft_consensus.cc:3037] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.535974 27449 raft_consensus.cc:3037] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.536482 27449 raft_consensus.cc:2454] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.536482 27585 raft_consensus.cc:2454] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.536633 27656 leader_election.cc:302] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.536700 28740 raft_consensus.cc:2781] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.536767 28740 raft_consensus.cc:686] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.536813 28740 consensus_queue.cc:237] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.537395 27198 catalog_manager.cc:5484] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
W20240503 02:09:14.553982 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:14.577114 28685 raft_consensus.cc:482] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.577168 28685 raft_consensus.cc:504] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.577275 28409 raft_consensus.cc:482] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.577311 28685 leader_election.cc:288] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.577414 28409 raft_consensus.cc:504] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.577469 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7addc5958f04d1e83f77839e92201c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.577523 28409 leader_election.cc:288] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.577527 27721 raft_consensus.cc:2454] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.577535 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7addc5958f04d1e83f77839e92201c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.577586 27585 raft_consensus.cc:2454] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.577702 27388 leader_election.cc:302] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.577713 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b36a3207ff43c185993f0e9ea457fa" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.577764 27721 raft_consensus.cc:2454] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.577775 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b36a3207ff43c185993f0e9ea457fa" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.577817 27585 raft_consensus.cc:2454] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.577842 28409 raft_consensus.cc:2781] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.577872 28409 raft_consensus.cc:482] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.577890 28409 raft_consensus.cc:3037] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.577924 27388 leader_election.cc:302] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.578056 28685 raft_consensus.cc:2781] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.578084 28685 raft_consensus.cc:482] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.578106 28685 raft_consensus.cc:3037] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.578186 28740 raft_consensus.cc:482] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.578222 28740 raft_consensus.cc:504] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.578310 28740 leader_election.cc:288] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.578465 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a234271dc14a04a6feeb30e2fcd8f2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.578526 27585 raft_consensus.cc:2454] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.578636 27656 leader_election.cc:302] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.578640 28409 raft_consensus.cc:504] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.578696 28740 raft_consensus.cc:2781] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.578719 28740 raft_consensus.cc:482] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.578735 28740 raft_consensus.cc:3037] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.578747 28409 leader_election.cc:288] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.578789 28685 raft_consensus.cc:504] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.578878 28685 leader_election.cc:288] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.578935 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7addc5958f04d1e83f77839e92201c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.578975 27721 raft_consensus.cc:3037] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.579286 28740 raft_consensus.cc:504] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.579397 28740 leader_election.cc:288] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.579469 27721 raft_consensus.cc:2454] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.579524 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b36a3207ff43c185993f0e9ea457fa" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.579537 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56b36a3207ff43c185993f0e9ea457fa" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.579571 27587 raft_consensus.cc:3037] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.579576 27722 raft_consensus.cc:3037] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.579596 27388 leader_election.cc:302] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.579833 28685 raft_consensus.cc:2781] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.579893 28685 raft_consensus.cc:686] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.579934 28685 consensus_queue.cc:237] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.580106 27722 raft_consensus.cc:2454] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.580106 27587 raft_consensus.cc:2454] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.580191 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a234271dc14a04a6feeb30e2fcd8f2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.580185 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a234271dc14a04a6feeb30e2fcd8f2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.580242 27582 raft_consensus.cc:3037] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.580240 27449 raft_consensus.cc:2454] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.580493 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7addc5958f04d1e83f77839e92201c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.580543 27585 raft_consensus.cc:3037] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.580646 27388 leader_election.cc:302] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.581059 27585 raft_consensus.cc:2454] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.581079 28409 raft_consensus.cc:2781] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.581117 28409 raft_consensus.cc:686] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.581123 27582 raft_consensus.cc:2454] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.581151 28409 consensus_queue.cc:237] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.581178 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a234271dc14a04a6feeb30e2fcd8f2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.581230 27449 raft_consensus.cc:3037] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.581658 27656 leader_election.cc:302] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.581787 27449 raft_consensus.cc:2454] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.581848 28740 raft_consensus.cc:2781] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.582180 28740 raft_consensus.cc:686] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.582235 28740 consensus_queue.cc:237] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.582326 27198 catalog_manager.cc:5484] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.584450 27198 catalog_manager.cc:5484] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.584551 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.585013 27197 catalog_manager.cc:5484] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.587430 28507 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.587489 28507 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.587625 28507 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:14.587960 27523 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
W20240503 02:09:14.588088 27524 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
I20240503 02:09:14.588120 27524 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:14.588178 28507 raft_consensus.cc:2726] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:14.590548 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3d13fc887dfc4a33a3c4729168d769b3" } modify_external_catalogs: true
I20240503 02:09:14.590621 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3d13fc887dfc4a33a3c4729168d769b3" } modify_external_catalogs: true
I20240503 02:09:14.591714 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a12a765f85d43f180a9ce0254c2fd10 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.591771 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet db95b9847807414aa63e686837a0c9c1
I20240503 02:09:14.592012 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet db95b9847807414aa63e686837a0c9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.592026 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet db95b9847807414aa63e686837a0c9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.592090 28654 tablet_replica.cc:330] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.592111 28661 tablet_replica.cc:330] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.592140 28654 raft_consensus.cc:2229] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.592176 28654 raft_consensus.cc:2258] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.592176 28661 raft_consensus.cc:2229] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.592264 28654 ts_tablet_manager.cc:1875] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.592262 28661 raft_consensus.cc:2258] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.592391 28661 ts_tablet_manager.cc:1875] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.593259 28654 ts_tablet_manager.cc:1888] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.593302 28654 log.cc:1196] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/db95b9847807414aa63e686837a0c9c1
I20240503 02:09:14.593520 28654 ts_tablet_manager.cc:1909] T db95b9847807414aa63e686837a0c9c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.593741 28661 ts_tablet_manager.cc:1888] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.593794 28661 log.cc:1196] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/db95b9847807414aa63e686837a0c9c1
I20240503 02:09:14.593950 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]) successfully deleted
I20240503 02:09:14.593974 28661 ts_tablet_manager.cc:1909] T db95b9847807414aa63e686837a0c9c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.594209 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bda4d7a547de45e187f89fe8ff79deff" } modify_external_catalogs: true
I20240503 02:09:14.594261 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1b5d5f311cf946068932b35db1021da1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.594312 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bda4d7a547de45e187f89fe8ff79deff" } modify_external_catalogs: true
W20240503 02:09:14.594363 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b5d5f311cf946068932b35db1021da1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.594496 27203 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_22e35ec738b34697afb7f6d08708a4b9" } modify_external_catalogs: true
I20240503 02:09:14.594529 27203 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_22e35ec738b34697afb7f6d08708a4b9" } modify_external_catalogs: true
I20240503 02:09:14.594209 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9017bb9c723a401c9008d8ef638838ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.594699 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9017bb9c723a401c9008d8ef638838ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.594713 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]) successfully deleted
I20240503 02:09:14.595746 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet e42df1b4d5b24fda8d1865c86f34d957 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.595813 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet e7addc5958f04d1e83f77839e92201c3 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.595893 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet db95b9847807414aa63e686837a0c9c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.595911 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f4b60bdcec7a46a0b573c65c264d75b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.595984 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4b60bdcec7a46a0b573c65c264d75b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.595976 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f6a1485d0eca4902b9511e707ab168dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.596063 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6a1485d0eca4902b9511e707ab168dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.596179 28653 tablet_replica.cc:330] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.596243 28653 raft_consensus.cc:2229] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.596277 28653 raft_consensus.cc:2258] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.596383 28653 ts_tablet_manager.cc:1875] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.596956 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet da16da72302e49568ea8d2967f0d2e7a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.597702 28653 ts_tablet_manager.cc:1888] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.597745 28653 log.cc:1196] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/db95b9847807414aa63e686837a0c9c1
I20240503 02:09:14.597857 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:14.597918 28653 ts_tablet_manager.cc:1909] T db95b9847807414aa63e686837a0c9c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.597940 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 3a12a765f85d43f180a9ce0254c2fd10 deleted and replaced by tablet d75acbf10cd547e6925caca235db8f46
I20240503 02:09:14.598003 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 9d1fbd4e5618451fac90622fbcf1e9bb from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.598098 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12a765f85d43f180a9ce0254c2fd10 with delete_type TABLET_DATA_DELETED (replaced by tablet d75acbf10cd547e6925caca235db8f46) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.598168 28661 tablet_replica.cc:330] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.598199 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12a765f85d43f180a9ce0254c2fd10 with delete_type TABLET_DATA_DELETED (replaced by tablet d75acbf10cd547e6925caca235db8f46) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.598227 28661 raft_consensus.cc:2229] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.598255 28661 raft_consensus.cc:2258] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.598268 27582 raft_consensus.cc:1261] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.598345 28661 ts_tablet_manager.cc:1875] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.598399 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a12a765f85d43f180a9ce0254c2fd10 with delete_type TABLET_DATA_DELETED (replaced by tablet d75acbf10cd547e6925caca235db8f46) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.598446 28653 tablet_replica.cc:330] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.598505 28653 raft_consensus.cc:2229] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.598568 28653 raft_consensus.cc:2258] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.598665 28653 ts_tablet_manager.cc:1875] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.599143 28654 tablet_replica.cc:330] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.599203 28654 raft_consensus.cc:2229] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.599231 28654 raft_consensus.cc:2258] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.599318 28654 ts_tablet_manager.cc:1875] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.599429 28661 ts_tablet_manager.cc:1888] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.599469 28661 log.cc:1196] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:14.599625 28661 ts_tablet_manager.cc:1909] T 3a12a765f85d43f180a9ce0254c2fd10 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.599684 28653 ts_tablet_manager.cc:1888] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.599726 28653 log.cc:1196] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:14.599845 28685 consensus_queue.cc:1035] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.600031 28653 ts_tablet_manager.cc:1909] T 3a12a765f85d43f180a9ce0254c2fd10 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.600072 27722 raft_consensus.cc:1261] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.600206 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet db95b9847807414aa63e686837a0c9c1 (table default.Table_3d13fc887dfc4a33a3c4729168d769b3 [id=5553fa58eac145c885b8f2325c4ef52d]) successfully deleted
I20240503 02:09:14.600304 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3a12a765f85d43f180a9ce0254c2fd10 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.600405 28654 ts_tablet_manager.cc:1888] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.600447 28654 log.cc:1196] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3a12a765f85d43f180a9ce0254c2fd10
I20240503 02:09:14.600620 28654 ts_tablet_manager.cc:1909] T 3a12a765f85d43f180a9ce0254c2fd10 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.600613 27203 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 00462246db1a4a53ba9f9a27a86ef538
I20240503 02:09:14.600838 28685 consensus_queue.cc:1035] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.600870 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e7addc5958f04d1e83f77839e92201c3
I20240503 02:09:14.600894 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e42df1b4d5b24fda8d1865c86f34d957
I20240503 02:09:14.600919 27200 catalog_manager.cc:6554] ReplaceTablet: tablet e7addc5958f04d1e83f77839e92201c3 deleted and replaced by tablet 9448d14b53f74e7f8ccbc19bb07b3a6d
I20240503 02:09:14.600943 27201 catalog_manager.cc:6554] ReplaceTablet: tablet e42df1b4d5b24fda8d1865c86f34d957 deleted and replaced by tablet 1ba4724c10854e468681870a4edda54b
I20240503 02:09:14.600991 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4ea3f123adc34f7db81d534fdb78c505
I20240503 02:09:14.601137 27696 tablet_service.cc:1514] Processing DeleteTablet for tablet e7addc5958f04d1e83f77839e92201c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 9448d14b53f74e7f8ccbc19bb07b3a6d) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.601197 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ba4724c10854e468681870a4edda54b) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.601246 27695 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.601307 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.601711 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.601797 27194 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7a4dd759006b4f999a36751c7b62ae48" } new_table_name: "default.Table_e23e16bec54548cbbbb515d698ffb45e" modify_external_catalogs: true
I20240503 02:09:14.601822 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9eebf349071044e7a8b8207e8e7dad3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.601874 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet e7addc5958f04d1e83f77839e92201c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 9448d14b53f74e7f8ccbc19bb07b3a6d) from {username='slave'} at 127.0.0.1:42360
W20240503 02:09:14.601909 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9eebf349071044e7a8b8207e8e7dad3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.601925 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ba4724c10854e468681870a4edda54b) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.601979 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.601963 27203 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8479162a1ff24fad8162a05b6165e50b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.602030 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
W20240503 02:09:14.602041 27203 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8479162a1ff24fad8162a05b6165e50b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.602087 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_dc6c9f67f3dc4f2cb042279e09b365a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.602144 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e30014f1b56649c080ec356295587b82" } new_table_name: "default.Table_3caec400749841ad8ba3c6822d3677ab" modify_external_catalogs: true
W20240503 02:09:14.602161 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc6c9f67f3dc4f2cb042279e09b365a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.601801 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1fc11241cf4f4a228e9bc8d96114976a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.602260 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e1e149474317465db7cfab2add6adb8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.602305 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1e149474317465db7cfab2add6adb8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.602332 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_efc4628674244e0a892ce7a48f15c83b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.602308 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fc11241cf4f4a228e9bc8d96114976a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:14.602370 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efc4628674244e0a892ce7a48f15c83b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.603071 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 00462246db1a4a53ba9f9a27a86ef538 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.603152 28653 tablet_replica.cc:330] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.603183 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet e42df1b4d5b24fda8d1865c86f34d957 with delete_type TABLET_DATA_DELETED (replaced by tablet 1ba4724c10854e468681870a4edda54b) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.603206 28653 raft_consensus.cc:2229] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.603257 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ea3f123adc34f7db81d534fdb78c505 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.603250 28653 raft_consensus.cc:2258] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.603313 28661 tablet_replica.cc:330] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.603318 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet e7addc5958f04d1e83f77839e92201c3 with delete_type TABLET_DATA_DELETED (replaced by tablet 9448d14b53f74e7f8ccbc19bb07b3a6d) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.603354 28661 raft_consensus.cc:2229] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.603374 28661 raft_consensus.cc:2258] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.603435 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3a12a765f85d43f180a9ce0254c2fd10 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.603456 28653 ts_tablet_manager.cc:1875] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.603543 28661 ts_tablet_manager.cc:1875] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.604166 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9d1fbd4e5618451fac90622fbcf1e9bb
I20240503 02:09:14.604246 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 9d1fbd4e5618451fac90622fbcf1e9bb deleted and replaced by tablet 80e4ce547683468a9cb0e1683b248508
I20240503 02:09:14.604291 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f4b60bdcec7a46a0b573c65c264d75b2" } new_table_name: "default.Table_21661264ffe84bba9777c3f8f8d15678" modify_external_catalogs: true
I20240503 02:09:14.604558 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 80e4ce547683468a9cb0e1683b248508) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.604610 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 80e4ce547683468a9cb0e1683b248508) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.604648 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1fbd4e5618451fac90622fbcf1e9bb with delete_type TABLET_DATA_DELETED (replaced by tablet 80e4ce547683468a9cb0e1683b248508) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.605080 28661 ts_tablet_manager.cc:1888] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.605121 28661 log.cc:1196] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e7addc5958f04d1e83f77839e92201c3
I20240503 02:09:14.605258 28661 ts_tablet_manager.cc:1909] T e7addc5958f04d1e83f77839e92201c3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.605461 28661 tablet_replica.cc:330] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.605509 28661 raft_consensus.cc:2229] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.605535 28661 raft_consensus.cc:2258] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.605942 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3954c32663f14e4b8cc0122d92f3c9cd" } new_table_name: "default.Table_63ccfec42f064aacaf0c950ec3442742" modify_external_catalogs: true
I20240503 02:09:14.606077 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 1f4bf62a461048fea8dda930b82f464e (DEFAULT_TABLE table=default.Table_1b5d5f311cf946068932b35db1021da1 [id=b70cacd384524dd890671729a21e4fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.606134 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e7addc5958f04d1e83f77839e92201c3 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:14.606169 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f4bf62a461048fea8dda930b82f464e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.606688 28661 ts_tablet_manager.cc:1875] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.607241 28656 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.607276 27565 tablet_service.cc:1467] Processing CreateTablet for tablet d75acbf10cd547e6925caca235db8f46 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.607357 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75acbf10cd547e6925caca235db8f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.607529 28661 ts_tablet_manager.cc:1888] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.606277 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1b5d5f311cf946068932b35db1021da1" } new_table_name: "default.Table_c24ee498ea4c4aaea463867f33b84d71" modify_external_catalogs: true
I20240503 02:09:14.607570 28661 log.cc:1196] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e42df1b4d5b24fda8d1865c86f34d957
I20240503 02:09:14.607623 28656 tablet_bootstrap.cc:654] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.607707 28661 ts_tablet_manager.cc:1909] T e42df1b4d5b24fda8d1865c86f34d957 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.607831 28685 raft_consensus.cc:482] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.607870 28685 raft_consensus.cc:504] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.607970 28685 leader_election.cc:288] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.608029 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 56b36a3207ff43c185993f0e9ea457fa (table default.Table_dbc2c33b084947c7854b5676739b95df [id=f62f02a2dfdd454e9ffdfe34656eaac3]). Expected version 1 got 0
I20240503 02:09:14.608125 27428 tablet_service.cc:1467] Processing CreateTablet for tablet d75acbf10cd547e6925caca235db8f46 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.608207 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75acbf10cd547e6925caca235db8f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.608263 28661 tablet_replica.cc:330] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.608304 28661 raft_consensus.cc:2229] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.608331 28661 raft_consensus.cc:2258] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.608430 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 1f4bf62a461048fea8dda930b82f464e (DEFAULT_TABLE table=default.Table_1b5d5f311cf946068932b35db1021da1 [id=b70cacd384524dd890671729a21e4fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.608521 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f4bf62a461048fea8dda930b82f464e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.608575 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c4e5ba1b274030b922c2234b0af7f3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.608631 27582 raft_consensus.cc:2454] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.608656 28661 ts_tablet_manager.cc:1875] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.609005 28653 ts_tablet_manager.cc:1888] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.609046 28653 log.cc:1196] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/00462246db1a4a53ba9f9a27a86ef538
I20240503 02:09:14.609190 28653 ts_tablet_manager.cc:1909] T 00462246db1a4a53ba9f9a27a86ef538 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.609443 28655 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.609647 28661 ts_tablet_manager.cc:1888] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.609688 28661 log.cc:1196] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4ea3f123adc34f7db81d534fdb78c505
I20240503 02:09:14.609791 28655 tablet_bootstrap.cc:654] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.609845 28661 ts_tablet_manager.cc:1909] T 4ea3f123adc34f7db81d534fdb78c505 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.610046 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e42df1b4d5b24fda8d1865c86f34d957 (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.610101 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c4e5ba1b274030b922c2234b0af7f3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.610177 28740 raft_consensus.cc:482] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.610153 27695 tablet_service.cc:1467] Processing CreateTablet for tablet 1f4bf62a461048fea8dda930b82f464e (DEFAULT_TABLE table=default.Table_1b5d5f311cf946068932b35db1021da1 [id=b70cacd384524dd890671729a21e4fcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.610222 28740 raft_consensus.cc:504] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.610271 28655 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.610276 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f4bf62a461048fea8dda930b82f464e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.610304 28655 ts_tablet_manager.cc:1367] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.610309 28740 leader_election.cc:288] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.610399 27386 leader_election.cc:302] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.610116 27700 tablet_service.cc:1467] Processing CreateTablet for tablet d75acbf10cd547e6925caca235db8f46 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.610435 28655 raft_consensus.cc:348] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.610482 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d75acbf10cd547e6925caca235db8f46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.610483 28655 raft_consensus.cc:374] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.610512 28655 raft_consensus.cc:729] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.610525 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685ad701b1c143d4a77f6ec0c738c125" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.610549 28655 consensus_queue.cc:260] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.610602 28685 raft_consensus.cc:2781] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.610620 28685 raft_consensus.cc:482] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.610636 28685 raft_consensus.cc:3037] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.610643 28655 ts_tablet_manager.cc:1398] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.610705 28655 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.611011 28655 tablet_bootstrap.cc:654] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.611184 28685 raft_consensus.cc:504] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.611289 28685 leader_election.cc:288] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.610165 27721 raft_consensus.cc:2454] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.611403 27451 raft_consensus.cc:1261] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.610571 27449 raft_consensus.cc:2454] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.611533 28664 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.611747 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c4e5ba1b274030b922c2234b0af7f3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.611805 27721 raft_consensus.cc:3037] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.611862 28664 tablet_bootstrap.cc:654] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.611897 27659 leader_election.cc:302] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.611956 28693 consensus_queue.cc:1035] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.611994 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c4e5ba1b274030b922c2234b0af7f3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.612026 28693 raft_consensus.cc:2781] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.612046 28693 raft_consensus.cc:482] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.612046 27582 raft_consensus.cc:3037] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.612061 28693 raft_consensus.cc:3037] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.612346 27721 raft_consensus.cc:2454] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.612581 27582 raft_consensus.cc:2454] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.612588 28693 raft_consensus.cc:504] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.612686 28693 leader_election.cc:288] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.612680 27583 raft_consensus.cc:1261] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.612754 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685ad701b1c143d4a77f6ec0c738c125" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.612794 27587 raft_consensus.cc:2454] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.612843 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685ad701b1c143d4a77f6ec0c738c125" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.612877 27583 raft_consensus.cc:3037] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.612910 27853 consensus_queue.cc:1035] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.613073 27388 leader_election.cc:302] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.613581 28685 raft_consensus.cc:2781] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.613621 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685ad701b1c143d4a77f6ec0c738c125" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.613682 27453 raft_consensus.cc:3037] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.613692 28685 raft_consensus.cc:686] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.613735 28685 consensus_queue.cc:237] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.614286 27453 raft_consensus.cc:2454] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.614430 27659 leader_election.cc:302] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.614439 28710 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.614499 27853 raft_consensus.cc:2781] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.614542 27853 raft_consensus.cc:686] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.614576 27853 consensus_queue.cc:237] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.614709 28661 tablet_replica.cc:330] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.614835 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:14.614861 28698 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:14.615105 28661 raft_consensus.cc:2229] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.615144 28661 raft_consensus.cc:2258] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.615329 28661 ts_tablet_manager.cc:1875] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.615545 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet da16da72302e49568ea8d2967f0d2e7a
I20240503 02:09:14.615620 27198 catalog_manager.cc:6554] ReplaceTablet: tablet da16da72302e49568ea8d2967f0d2e7a deleted and replaced by tablet 59299d27526640828eda00961363d0a6
I20240503 02:09:14.615685 27194 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_99f1ffda145649899d3aa39d2147bdd4" } modify_external_catalogs: true
I20240503 02:09:14.615743 27194 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_99f1ffda145649899d3aa39d2147bdd4" } modify_external_catalogs: true
I20240503 02:09:14.615850 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet da16da72302e49568ea8d2967f0d2e7a with delete_type TABLET_DATA_DELETED (replaced by tablet 59299d27526640828eda00961363d0a6) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.616084 28706 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:14.616189 28654 tablet_replica.cc:330] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.616238 28654 raft_consensus.cc:2229] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.616264 28654 raft_consensus.cc:2258] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.616514 28655 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.616550 28655 ts_tablet_manager.cc:1367] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:14.616672 28661 ts_tablet_manager.cc:1888] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.616665 28655 raft_consensus.cc:348] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.616712 28655 raft_consensus.cc:374] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.616716 28661 log.cc:1196] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/00462246db1a4a53ba9f9a27a86ef538
I20240503 02:09:14.616731 28655 raft_consensus.cc:729] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.616777 28655 consensus_queue.cc:260] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.616850 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3a12a765f85d43f180a9ce0254c2fd10 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.616860 28655 ts_tablet_manager.cc:1398] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.616890 28661 ts_tablet_manager.cc:1909] T 00462246db1a4a53ba9f9a27a86ef538 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.616909 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet da16da72302e49568ea8d2967f0d2e7a with delete_type TABLET_DATA_DELETED (replaced by tablet 59299d27526640828eda00961363d0a6) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.617169 28656 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.617194 27583 raft_consensus.cc:2454] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.617205 28656 ts_tablet_manager.cc:1367] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:14.617296 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 00462246db1a4a53ba9f9a27a86ef538 (table default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]) successfully deleted
I20240503 02:09:14.617338 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_390a24b35e2f40f982412479ea7ebaec" } modify_external_catalogs: true
I20240503 02:09:14.617354 28656 raft_consensus.cc:348] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.617389 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_390a24b35e2f40f982412479ea7ebaec" } modify_external_catalogs: true
I20240503 02:09:14.617404 28656 raft_consensus.cc:374] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.617422 28656 raft_consensus.cc:729] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.617442 28661 tablet_replica.cc:330] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.617460 28656 consensus_queue.cc:260] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.617491 28661 raft_consensus.cc:2229] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.617532 28656 ts_tablet_manager.cc:1398] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.617539 28661 raft_consensus.cc:2258] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.617575 28656 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.617882 28661 ts_tablet_manager.cc:1875] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.617903 28656 tablet_bootstrap.cc:654] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.618080 28686 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:14.618505 28654 ts_tablet_manager.cc:1875] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.618518 27853 raft_consensus.cc:482] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.618562 27853 raft_consensus.cc:504] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.618640 27853 leader_election.cc:288] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.619083 28661 ts_tablet_manager.cc:1888] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.619127 28661 log.cc:1196] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9d1fbd4e5618451fac90622fbcf1e9bb
I20240503 02:09:14.619279 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b46bb7a647c64572bcae04d4dcdc1cda" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.619338 27451 raft_consensus.cc:2454] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.619298 28661 ts_tablet_manager.cc:1909] T 9d1fbd4e5618451fac90622fbcf1e9bb P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.619748 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:14.619825 27659 leader_election.cc:302] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.619894 27853 raft_consensus.cc:2781] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.619916 27853 raft_consensus.cc:482] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.619931 27853 raft_consensus.cc:3037] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.619940 28661 tablet_replica.cc:330] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.619989 28661 raft_consensus.cc:2229] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.620020 28661 raft_consensus.cc:2258] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.620105 28661 ts_tablet_manager.cc:1875] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240503 02:09:14.620487 27197 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 has no consensus state
I20240503 02:09:14.620512 27853 raft_consensus.cc:504] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.620615 27853 leader_election.cc:288] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.621598 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 80e4ce547683468a9cb0e1683b248508 (DEFAULT_TABLE table=default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.621627 27700 tablet_service.cc:1467] Processing CreateTablet for tablet a571596f97cd4fcbb9247faaed98bf7c (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.621677 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e4ce547683468a9cb0e1683b248508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.621713 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a571596f97cd4fcbb9247faaed98bf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.621749 27696 tablet_service.cc:1467] Processing CreateTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc (DEFAULT_TABLE table=default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.621832 27696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d002acb8924f4d2194ed1e5f462ebfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.622081 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 80e4ce547683468a9cb0e1683b248508 (DEFAULT_TABLE table=default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622165 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc (DEFAULT_TABLE table=default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622241 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 503bd37c6def464496f6dd4489fca009 (DEFAULT_TABLE table=default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622303 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622485 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (DEFAULT_TABLE table=default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622563 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a571596f97cd4fcbb9247faaed98bf7c (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.622777 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b46bb7a647c64572bcae04d4dcdc1cda" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.622828 27582 raft_consensus.cc:3037] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.622824 27698 tablet_service.cc:1467] Processing CreateTablet for tablet 503bd37c6def464496f6dd4489fca009 (DEFAULT_TABLE table=default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]), partition=RANGE (key) PARTITION UNBOUNDED
W20240503 02:09:14.622864 27200 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f4bf62a461048fea8dda930b82f464e consensus state has no leader
I20240503 02:09:14.622905 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 503bd37c6def464496f6dd4489fca009. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623165 28698 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.623335 27582 raft_consensus.cc:2454] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.623536 27656 leader_election.cc:302] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.623607 28654 ts_tablet_manager.cc:1888] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.623625 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623643 28654 log.cc:1196] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e7addc5958f04d1e83f77839e92201c3
I20240503 02:09:14.623715 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d002acb8924f4d2194ed1e5f462ebfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623778 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 503bd37c6def464496f6dd4489fca009. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623852 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e4ce547683468a9cb0e1683b248508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623868 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.623908 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a571596f97cd4fcbb9247faaed98bf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623950 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.623976 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.624846 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (DEFAULT_TABLE table=default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.624925 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.625062 28656 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.625099 28656 ts_tablet_manager.cc:1367] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:14.625190 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3caec400749841ad8ba3c6822d3677ab [id=b2a8365635134977a758514aa7a4351d] alter complete (version 3)
I20240503 02:09:14.625213 28656 raft_consensus.cc:348] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.625262 28656 raft_consensus.cc:374] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.625280 28656 raft_consensus.cc:729] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.625312 28656 consensus_queue.cc:260] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.625437 28656 ts_tablet_manager.cc:1398] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.625485 28656 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.625810 28656 tablet_bootstrap.cc:654] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.625883 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba4724c10854e468681870a4edda54b (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.625938 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba4724c10854e468681870a4edda54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.625939 27203 catalog_manager.cc:5484] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.626051 27203 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]). Expected version 1 got 0
I20240503 02:09:14.626539 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.626585 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2e99d982dddd497988d2d7220b96aefa (table default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f]). Expected version 1 got 0
I20240503 02:09:14.626597 27554 tablet_service.cc:1467] Processing CreateTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.626677 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51af2f65424fb0a7d26e13e4cd72ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.626708 27553 tablet_service.cc:1467] Processing CreateTablet for tablet 0afe93270c134207a6845c1e49de1b4d (DEFAULT_TABLE table=default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.626796 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0afe93270c134207a6845c1e49de1b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.626935 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba4724c10854e468681870a4edda54b (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.627013 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba4724c10854e468681870a4edda54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.627902 27555 tablet_service.cc:1467] Processing CreateTablet for tablet c351b66df551435ea01295f1f8dabd25 (DEFAULT_TABLE table=default.Table_dc6c9f67f3dc4f2cb042279e09b365a6 [id=ec2a47b26ace411681d838a53cad9bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.627964 27555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c351b66df551435ea01295f1f8dabd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.628108 28693 raft_consensus.cc:2781] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.628175 28693 raft_consensus.cc:686] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.628211 28693 consensus_queue.cc:237] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.628731 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 685ad701b1c143d4a77f6ec0c738c125
I20240503 02:09:14.629022 27556 tablet_service.cc:1467] Processing CreateTablet for tablet 59299d27526640828eda00961363d0a6 (DEFAULT_TABLE table=default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.629072 27703 tablet_service.cc:1467] Processing CreateTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.629110 27556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59299d27526640828eda00961363d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.629158 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51af2f65424fb0a7d26e13e4cd72ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.629200 27696 tablet_service.cc:1467] Processing CreateTablet for tablet 59299d27526640828eda00961363d0a6 (DEFAULT_TABLE table=default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.629262 27696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59299d27526640828eda00961363d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.629837 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 6149f14895e34ea5994e3cbb94cf0bb4
I20240503 02:09:14.629161 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 0afe93270c134207a6845c1e49de1b4d (DEFAULT_TABLE table=default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.630165 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffa71e4ead479b801922f8a385be4e (DEFAULT_TABLE table=default.Table_8479162a1ff24fad8162a05b6165e50b [id=3fc56404e8894e06afad9ccc8d82cf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.630192 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0afe93270c134207a6845c1e49de1b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.630221 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffa71e4ead479b801922f8a385be4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.630259 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (DEFAULT_TABLE table=default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.630273 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffa71e4ead479b801922f8a385be4e (DEFAULT_TABLE table=default.Table_8479162a1ff24fad8162a05b6165e50b [id=3fc56404e8894e06afad9ccc8d82cf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.630326 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.630363 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffa71e4ead479b801922f8a385be4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.631160 27695 tablet_service.cc:1514] Processing DeleteTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.631198 27698 tablet_service.cc:1467] Processing CreateTablet for tablet c351b66df551435ea01295f1f8dabd25 (DEFAULT_TABLE table=default.Table_dc6c9f67f3dc4f2cb042279e09b365a6 [id=ec2a47b26ace411681d838a53cad9bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.631263 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c351b66df551435ea01295f1f8dabd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.631398 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.626681 27552 tablet_service.cc:1467] Processing CreateTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (DEFAULT_TABLE table=default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.631577 28706 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.631620 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.631691 28704 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.632974 27194 catalog_manager.cc:5484] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.633872 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b46bb7a647c64572bcae04d4dcdc1cda" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.633941 27451 raft_consensus.cc:3037] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.634464 28661 ts_tablet_manager.cc:1888] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.634508 28661 log.cc:1196] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/da16da72302e49568ea8d2967f0d2e7a
I20240503 02:09:14.634668 28661 ts_tablet_manager.cc:1909] T da16da72302e49568ea8d2967f0d2e7a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.634964 28661 tablet_replica.cc:330] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.635061 28661 raft_consensus.cc:2229] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.635133 28661 raft_consensus.cc:2258] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.635188 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:14.635329 28661 ts_tablet_manager.cc:1875] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.636152 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b46bb7a647c64572bcae04d4dcdc1cda" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.636207 27583 raft_consensus.cc:2362] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:14.636422 28661 ts_tablet_manager.cc:1888] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.636528 28661 log.cc:1196] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/6149f14895e34ea5994e3cbb94cf0bb4
I20240503 02:09:14.636665 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.636866 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.636891 28661 ts_tablet_manager.cc:1909] T 6149f14895e34ea5994e3cbb94cf0bb4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.637066 28661 tablet_replica.cc:330] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.637154 28661 raft_consensus.cc:2229] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.623783 28654 ts_tablet_manager.cc:1909] T e7addc5958f04d1e83f77839e92201c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.637235 28661 pending_rounds.cc:62] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:14.637259 28661 pending_rounds.cc:68] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025301565440 op_type: NO_OP noop_request { }
I20240503 02:09:14.637290 28661 raft_consensus.cc:2866] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.637308 28661 raft_consensus.cc:2258] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.637437 28654 tablet_replica.cc:330] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.637488 28654 raft_consensus.cc:2229] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.637524 28654 pending_rounds.cc:62] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:14.637540 28654 pending_rounds.cc:68] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420024930643968 op_type: NO_OP noop_request { }
I20240503 02:09:14.637571 28654 raft_consensus.cc:2866] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.637590 28654 raft_consensus.cc:2258] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.637650 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet da16da72302e49568ea8d2967f0d2e7a with delete_type TABLET_DATA_DELETED (replaced by tablet 59299d27526640828eda00961363d0a6) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.637722 27430 tablet_service.cc:1467] Processing CreateTablet for tablet 80e4ce547683468a9cb0e1683b248508 (DEFAULT_TABLE table=default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.637795 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e4ce547683468a9cb0e1683b248508. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.637811 27428 tablet_service.cc:1467] Processing CreateTablet for tablet a571596f97cd4fcbb9247faaed98bf7c (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.637907 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a571596f97cd4fcbb9247faaed98bf7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.637953 28653 tablet_replica.cc:330] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.638000 28653 raft_consensus.cc:2229] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.638022 28653 raft_consensus.cc:2258] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.638557 27451 raft_consensus.cc:2454] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.638624 28664 tablet_bootstrap.cc:492] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.638662 28664 ts_tablet_manager.cc:1367] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20240503 02:09:14.638661 28685 raft_consensus.cc:482] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.638692 28685 raft_consensus.cc:504] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.638761 28685 leader_election.cc:288] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.638792 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 503bd37c6def464496f6dd4489fca009 (DEFAULT_TABLE table=default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.638808 28664 raft_consensus.cc:348] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.638859 28664 raft_consensus.cc:374] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.638862 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 503bd37c6def464496f6dd4489fca009. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.638876 28664 raft_consensus.cc:729] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.638882 28653 ts_tablet_manager.cc:1875] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.638907 28664 consensus_queue.cc:260] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.638988 28664 ts_tablet_manager.cc:1398] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.639007 27433 tablet_service.cc:1467] Processing CreateTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc (DEFAULT_TABLE table=default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.639045 28664 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.639077 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d002acb8924f4d2194ed1e5f462ebfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.639358 28664 tablet_bootstrap.cc:654] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.639868 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.639997 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 6149f14895e34ea5994e3cbb94cf0bb4 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.640017 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (DEFAULT_TABLE table=default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640061 28664 tablet_bootstrap.cc:492] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.640089 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba4724c10854e468681870a4edda54b (DEFAULT_TABLE table=default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640100 28664 ts_tablet_manager.cc:1367] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.640127 27425 tablet_service.cc:1467] Processing CreateTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640165 27431 tablet_service.cc:1467] Processing CreateTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (DEFAULT_TABLE table=default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640224 27423 tablet_service.cc:1467] Processing CreateTablet for tablet 0afe93270c134207a6845c1e49de1b4d (DEFAULT_TABLE table=default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640223 28664 raft_consensus.cc:348] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.640281 28664 raft_consensus.cc:374] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.640300 28664 raft_consensus.cc:729] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.640312 27423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0afe93270c134207a6845c1e49de1b4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.640329 28664 consensus_queue.cc:260] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.640349 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75acbf10cd547e6925caca235db8f46" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.640422 27721 raft_consensus.cc:2454] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.640413 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 00462246db1a4a53ba9f9a27a86ef538 (table default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]) successfully deleted
I20240503 02:09:14.640441 28664 ts_tablet_manager.cc:1398] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.640229 28655 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.640491 28664 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.640501 27421 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffa71e4ead479b801922f8a385be4e (DEFAULT_TABLE table=default.Table_8479162a1ff24fad8162a05b6165e50b [id=3fc56404e8894e06afad9ccc8d82cf89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640578 27420 tablet_service.cc:1514] Processing DeleteTablet for tablet 685ad701b1c143d4a77f6ec0c738c125 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.640605 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba4724c10854e468681870a4edda54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.640587 27388 leader_election.cc:302] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.640661 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet 6149f14895e34ea5994e3cbb94cf0bb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.640689 28685 raft_consensus.cc:2781] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.640714 28685 raft_consensus.cc:482] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.640723 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc51af2f65424fb0a7d26e13e4cd72ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.640722 27424 tablet_service.cc:1467] Processing CreateTablet for tablet c351b66df551435ea01295f1f8dabd25 (DEFAULT_TABLE table=default.Table_dc6c9f67f3dc4f2cb042279e09b365a6 [id=ec2a47b26ace411681d838a53cad9bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640780 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.640784 28664 tablet_bootstrap.cc:654] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.640731 28685 raft_consensus.cc:3037] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.640463 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 59299d27526640828eda00961363d0a6 (DEFAULT_TABLE table=default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.640914 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.641213 28706 tablet.cc:1722] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.641397 28685 raft_consensus.cc:504] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.641491 28685 leader_election.cc:288] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.641799 28664 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.641834 28664 ts_tablet_manager.cc:1367] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.641947 28664 raft_consensus.cc:348] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.641994 28664 raft_consensus.cc:374] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.641978 27431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.642014 28664 raft_consensus.cc:729] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.642038 28664 consensus_queue.cc:260] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.642107 28664 ts_tablet_manager.cc:1398] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.642153 28664 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.642195 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e7addc5958f04d1e83f77839e92201c3 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:14.642454 28664 tablet_bootstrap.cc:654] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.642532 27848 raft_consensus.cc:482] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.642572 27848 raft_consensus.cc:504] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.640316 28653 ts_tablet_manager.cc:1888] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.642654 27848 leader_election.cc:288] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.642663 28653 log.cc:1196] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e42df1b4d5b24fda8d1865c86f34d957
I20240503 02:09:14.642809 28653 ts_tablet_manager.cc:1909] T e42df1b4d5b24fda8d1865c86f34d957 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.642810 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75acbf10cd547e6925caca235db8f46" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.642859 27587 raft_consensus.cc:2454] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.642956 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a8e43c8d5554686aa9ecee00d265c92" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.642988 28653 tablet_replica.cc:330] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.643002 27449 raft_consensus.cc:2454] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.643127 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e42df1b4d5b24fda8d1865c86f34d957 (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.643214 28661 ts_tablet_manager.cc:1875] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.643249 27524 leader_election.cc:302] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.643290 28653 raft_consensus.cc:2229] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.643323 27848 raft_consensus.cc:2781] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.643298 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75acbf10cd547e6925caca235db8f46" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.643345 27848 raft_consensus.cc:482] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.643352 27723 raft_consensus.cc:3037] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.643357 28653 raft_consensus.cc:2258] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.643361 27848 raft_consensus.cc:3037] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.643543 28653 ts_tablet_manager.cc:1875] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.643891 27723 raft_consensus.cc:2454] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.643891 27848 raft_consensus.cc:504] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.643991 27848 leader_election.cc:288] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.644485 27388 leader_election.cc:302] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.644490 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a8e43c8d5554686aa9ecee00d265c92" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.644546 28653 ts_tablet_manager.cc:1888] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.644569 27723 raft_consensus.cc:3037] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.644583 28653 log.cc:1196] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4ea3f123adc34f7db81d534fdb78c505
I20240503 02:09:14.644747 28653 ts_tablet_manager.cc:1909] T 4ea3f123adc34f7db81d534fdb78c505 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.644810 28685 raft_consensus.cc:2781] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.644851 28685 raft_consensus.cc:686] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.644881 28685 consensus_queue.cc:237] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.644990 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a8e43c8d5554686aa9ecee00d265c92" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.645042 27449 raft_consensus.cc:3037] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.645110 27723 raft_consensus.cc:2454] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.645205 28661 ts_tablet_manager.cc:1888] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.645234 28661 log.cc:1196] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/685ad701b1c143d4a77f6ec0c738c125
I20240503 02:09:14.645223 27523 leader_election.cc:302] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.645305 27203 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05] alter complete (version 2)
I20240503 02:09:14.645426 28661 ts_tablet_manager.cc:1909] T 685ad701b1c143d4a77f6ec0c738c125 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.645484 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.645526 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2e99d982dddd497988d2d7220b96aefa (table default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f]). Expected version 1 got 0
I20240503 02:09:14.645548 27194 catalog_manager.cc:5484] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.645603 27449 raft_consensus.cc:2454] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.646059 28653 tablet_replica.cc:330] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.646147 28653 raft_consensus.cc:2229] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.646183 28653 pending_rounds.cc:62] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:14.646198 28653 pending_rounds.cc:68] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025159610368 op_type: NO_OP noop_request { }
I20240503 02:09:14.646227 28653 raft_consensus.cc:2866] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.646246 28653 raft_consensus.cc:2258] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.646430 28653 ts_tablet_manager.cc:1875] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.646651 28710 tablet.cc:1722] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.646762 27848 raft_consensus.cc:2781] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.646811 27848 raft_consensus.cc:686] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.646842 27848 consensus_queue.cc:237] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.647679 28653 ts_tablet_manager.cc:1888] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.647722 28653 log.cc:1196] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e7addc5958f04d1e83f77839e92201c3
I20240503 02:09:14.647893 28655 tablet_bootstrap.cc:654] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.647897 28653 ts_tablet_manager.cc:1909] T e7addc5958f04d1e83f77839e92201c3 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.648345 28710 tablet.cc:1722] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.648437 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:14.648625 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffa71e4ead479b801922f8a385be4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.648730 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59299d27526640828eda00961363d0a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.648818 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c351b66df551435ea01295f1f8dabd25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.649137 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
I20240503 02:09:14.649358 28698 tablet.cc:1722] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.649395 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a8e43c8d5554686aa9ecee00d265c92" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.649461 27721 raft_consensus.cc:2362] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:14.649951 28655 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.649989 28655 ts_tablet_manager.cc:1367] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240503 02:09:14.650113 28655 raft_consensus.cc:348] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.650166 28655 raft_consensus.cc:374] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.650183 28655 raft_consensus.cc:729] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.650214 28655 consensus_queue.cc:260] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.650283 28655 ts_tablet_manager.cc:1398] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.650346 28655 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.650382 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.650655 28655 tablet_bootstrap.cc:654] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.650892 27203 catalog_manager.cc:5484] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:14.651854 28655 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.651890 28655 ts_tablet_manager.cc:1367] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.651997 28655 raft_consensus.cc:348] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.652046 28655 raft_consensus.cc:374] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.652062 28655 raft_consensus.cc:729] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.652099 28655 consensus_queue.cc:260] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.652194 28655 ts_tablet_manager.cc:1398] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.652249 28655 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.652575 28655 tablet_bootstrap.cc:654] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.652822 27194 catalog_manager.cc:5484] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.652912 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.653021 28655 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.653024 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f] alter complete (version 1)
I20240503 02:09:14.653059 28655 ts_tablet_manager.cc:1367] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.653198 28655 raft_consensus.cc:348] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.653245 28655 raft_consensus.cc:374] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.653260 28655 raft_consensus.cc:729] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.653553 28653 tablet_replica.cc:330] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.653601 28653 raft_consensus.cc:2229] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.653631 28653 raft_consensus.cc:2258] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.653292 28655 consensus_queue.cc:260] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.653720 28655 ts_tablet_manager.cc:1398] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.653787 28655 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.653846 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e7addc5958f04d1e83f77839e92201c3 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:14.653873 28653 ts_tablet_manager.cc:1875] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.654058 28655 tablet_bootstrap.cc:654] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.654729 28664 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.654767 28664 ts_tablet_manager.cc:1367] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:14.654822 28655 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.654858 28655 ts_tablet_manager.cc:1367] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.654902 28653 ts_tablet_manager.cc:1888] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.654896 28664 raft_consensus.cc:348] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.654938 28653 log.cc:1196] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9d1fbd4e5618451fac90622fbcf1e9bb
I20240503 02:09:14.654946 28664 raft_consensus.cc:374] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.654964 28664 raft_consensus.cc:729] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.654961 28655 raft_consensus.cc:348] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.654996 28655 raft_consensus.cc:374] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.655007 28655 raft_consensus.cc:729] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.655002 28664 consensus_queue.cc:260] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.655026 28655 consensus_queue.cc:260] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.655093 28664 ts_tablet_manager.cc:1398] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.655092 28655 ts_tablet_manager.cc:1398] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.655126 28653 ts_tablet_manager.cc:1909] T 9d1fbd4e5618451fac90622fbcf1e9bb P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.655140 28655 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.655148 28664 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.655444 28653 tablet_replica.cc:330] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.655462 28664 tablet_bootstrap.cc:654] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.655495 28653 raft_consensus.cc:2229] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.655529 28653 raft_consensus.cc:2258] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.655618 28653 ts_tablet_manager.cc:1875] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.655453 28655 tablet_bootstrap.cc:654] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.656162 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:14.656210 28655 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
W20240503 02:09:14.656459 27194 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 80e4ce547683468a9cb0e1683b248508 consensus state has no leader
I20240503 02:09:14.656246 28655 ts_tablet_manager.cc:1367] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.658849 28655 raft_consensus.cc:348] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.658916 28655 raft_consensus.cc:374] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.658936 28655 raft_consensus.cc:729] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.658968 28655 consensus_queue.cc:260] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.659041 28655 ts_tablet_manager.cc:1398] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.659096 28655 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.659401 28655 tablet_bootstrap.cc:654] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.657071 27203 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.656684 28653 ts_tablet_manager.cc:1888] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.659931 28653 log.cc:1196] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/da16da72302e49568ea8d2967f0d2e7a
I20240503 02:09:14.660117 28653 ts_tablet_manager.cc:1909] T da16da72302e49568ea8d2967f0d2e7a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.660297 28653 tablet_replica.cc:330] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.660339 28653 raft_consensus.cc:2229] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.660368 28653 raft_consensus.cc:2258] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.660475 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:14.660475 28655 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.660523 28655 ts_tablet_manager.cc:1367] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.660632 28655 raft_consensus.cc:348] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.660674 28655 raft_consensus.cc:374] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.660689 28655 raft_consensus.cc:729] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.660728 28655 consensus_queue.cc:260] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.660800 28655 ts_tablet_manager.cc:1398] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.660840 28655 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.661144 28655 tablet_bootstrap.cc:654] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.661533 28653 ts_tablet_manager.cc:1875] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.661625 28655 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.661655 28655 ts_tablet_manager.cc:1367] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.661770 28655 raft_consensus.cc:348] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.661818 28655 raft_consensus.cc:374] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.661832 28655 raft_consensus.cc:729] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.661866 28655 consensus_queue.cc:260] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.661949 28655 ts_tablet_manager.cc:1398] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.662003 28655 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.662298 28655 tablet_bootstrap.cc:654] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.662755 28653 ts_tablet_manager.cc:1888] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.663038 28653 log.cc:1196] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/685ad701b1c143d4a77f6ec0c738c125
I20240503 02:09:14.662866 28655 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.663118 28655 ts_tablet_manager.cc:1367] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.663247 28655 raft_consensus.cc:348] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.663311 28655 raft_consensus.cc:374] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.663327 28655 raft_consensus.cc:729] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.663352 28655 consensus_queue.cc:260] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.663416 28655 ts_tablet_manager.cc:1398] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.663482 28655 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.663605 28653 ts_tablet_manager.cc:1909] T 685ad701b1c143d4a77f6ec0c738c125 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.663825 28655 tablet_bootstrap.cc:654] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.664189 28653 tablet_replica.cc:330] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.664242 28653 raft_consensus.cc:2229] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.664281 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
I20240503 02:09:14.664264 28655 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.664319 28653 raft_consensus.cc:2258] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.664333 28655 ts_tablet_manager.cc:1367] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.664453 28653 ts_tablet_manager.cc:1875] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.664465 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d75acbf10cd547e6925caca235db8f46" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.664448 28655 raft_consensus.cc:348] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.664510 27583 raft_consensus.cc:3037] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.664515 28655 raft_consensus.cc:374] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.664533 28655 raft_consensus.cc:729] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.664572 28655 consensus_queue.cc:260] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.664642 28655 ts_tablet_manager.cc:1398] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.664688 28655 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.664968 28655 tablet_bootstrap.cc:654] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.665086 27583 raft_consensus.cc:2454] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.665587 28706 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.666043 28656 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.666081 28656 ts_tablet_manager.cc:1367] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.041s user 0.000s sys 0.000s
I20240503 02:09:14.666203 28656 raft_consensus.cc:348] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.666254 28656 raft_consensus.cc:374] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.666270 28656 raft_consensus.cc:729] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.666321 28656 consensus_queue.cc:260] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.666404 28656 ts_tablet_manager.cc:1398] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.666445 28656 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.666846 28656 tablet_bootstrap.cc:654] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.666919 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.667176 28653 ts_tablet_manager.cc:1888] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.667217 28653 log.cc:1196] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/6149f14895e34ea5994e3cbb94cf0bb4
I20240503 02:09:14.667357 28653 ts_tablet_manager.cc:1909] T 6149f14895e34ea5994e3cbb94cf0bb4 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.667693 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 6149f14895e34ea5994e3cbb94cf0bb4 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.668342 28654 ts_tablet_manager.cc:1875] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.669422 28654 ts_tablet_manager.cc:1888] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.669466 28654 log.cc:1196] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e42df1b4d5b24fda8d1865c86f34d957
I20240503 02:09:14.669576 28656 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.669623 28654 ts_tablet_manager.cc:1909] T e42df1b4d5b24fda8d1865c86f34d957 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.669627 28656 ts_tablet_manager.cc:1367] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:14.669764 28664 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.669797 28664 ts_tablet_manager.cc:1367] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20240503 02:09:14.669801 28654 tablet_replica.cc:330] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.669785 28656 raft_consensus.cc:348] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.669833 28656 raft_consensus.cc:374] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.669849 28654 raft_consensus.cc:2229] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.669853 28656 raft_consensus.cc:729] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.669910 28654 raft_consensus.cc:2258] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.669903 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e42df1b4d5b24fda8d1865c86f34d957 (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.669912 28664 raft_consensus.cc:348] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.669929 28656 consensus_queue.cc:260] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.669960 28664 raft_consensus.cc:374] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.669977 28664 raft_consensus.cc:729] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.670011 28664 consensus_queue.cc:260] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.670037 28656 ts_tablet_manager.cc:1398] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.670095 28656 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.670097 28664 ts_tablet_manager.cc:1398] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.670152 28664 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.670423 28656 tablet_bootstrap.cc:654] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.670586 28664 tablet_bootstrap.cc:654] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.670910 28656 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.670966 28656 ts_tablet_manager.cc:1367] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.671106 28656 raft_consensus.cc:348] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.671159 28656 raft_consensus.cc:374] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.671178 28656 raft_consensus.cc:729] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.671211 28656 consensus_queue.cc:260] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.671294 28656 ts_tablet_manager.cc:1398] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.671345 28656 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.671581 28655 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.671618 28655 ts_tablet_manager.cc:1367] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:14.671650 28656 tablet_bootstrap.cc:654] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.671723 28655 raft_consensus.cc:348] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.671772 28655 raft_consensus.cc:374] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.671790 28655 raft_consensus.cc:729] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.671830 28655 consensus_queue.cc:260] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.671905 28655 ts_tablet_manager.cc:1398] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.671970 28655 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.672017 28664 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.672060 28664 ts_tablet_manager.cc:1367] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.672195 28664 raft_consensus.cc:348] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.672250 28664 raft_consensus.cc:374] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.672271 28664 raft_consensus.cc:729] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.672284 28655 tablet_bootstrap.cc:654] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.672302 28664 consensus_queue.cc:260] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.672376 28664 ts_tablet_manager.cc:1398] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.672435 28664 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.672751 28664 tablet_bootstrap.cc:654] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.672781 28767 raft_consensus.cc:482] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.672832 28767 raft_consensus.cc:504] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.672839 27203 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.672932 28767 leader_election.cc:288] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.673147 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dabf28ec9f14b5998910b185852b22c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.673210 27722 raft_consensus.cc:2454] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.673305 28654 ts_tablet_manager.cc:1875] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.673527 27203 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.673676 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dabf28ec9f14b5998910b185852b22c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.673756 27583 raft_consensus.cc:2454] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.673882 28664 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.673918 28664 ts_tablet_manager.cc:1367] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.674036 28664 raft_consensus.cc:348] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.674088 28664 raft_consensus.cc:374] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.674106 28664 raft_consensus.cc:729] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.674142 28664 consensus_queue.cc:260] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.674237 28664 ts_tablet_manager.cc:1398] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.674280 28664 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.674587 28664 tablet_bootstrap.cc:654] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.674793 28654 ts_tablet_manager.cc:1888] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.674831 28654 log.cc:1196] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4ea3f123adc34f7db81d534fdb78c505
I20240503 02:09:14.674991 28654 ts_tablet_manager.cc:1909] T 4ea3f123adc34f7db81d534fdb78c505 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.675021 27388 leader_election.cc:302] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.675093 28767 raft_consensus.cc:2781] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.675122 28767 raft_consensus.cc:482] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.675138 28767 raft_consensus.cc:3037] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.675344 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4ea3f123adc34f7db81d534fdb78c505 (table default.Table_bda4d7a547de45e187f89fe8ff79deff [id=3995d190344c4d3f9d444cb266a9988a]) successfully deleted
I20240503 02:09:14.675426 28654 tablet_replica.cc:330] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.675472 28654 raft_consensus.cc:2229] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.675477 28664 tablet_bootstrap.cc:492] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.675503 28654 raft_consensus.cc:2258] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.675508 28664 ts_tablet_manager.cc:1367] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.675654 28654 ts_tablet_manager.cc:1875] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.675627 28664 raft_consensus.cc:348] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.675678 28664 raft_consensus.cc:374] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.675695 28664 raft_consensus.cc:729] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.675719 28664 consensus_queue.cc:260] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.675729 28767 raft_consensus.cc:504] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.675920 28664 ts_tablet_manager.cc:1398] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.675976 28664 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.676672 28654 ts_tablet_manager.cc:1888] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.676702 28664 tablet_bootstrap.cc:654] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.676787 28654 log.cc:1196] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/00462246db1a4a53ba9f9a27a86ef538
I20240503 02:09:14.676875 27203 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.676810 28656 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.676916 28767 leader_election.cc:288] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.676934 28656 ts_tablet_manager.cc:1367] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:14.677045 28656 raft_consensus.cc:348] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.677098 28656 raft_consensus.cc:374] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.677129 28656 raft_consensus.cc:729] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.677160 28656 consensus_queue.cc:260] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.677203 28655 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.677232 28656 ts_tablet_manager.cc:1398] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.677239 28655 ts_tablet_manager.cc:1367] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:14.677301 28656 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.677378 28655 raft_consensus.cc:348] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.677429 28655 raft_consensus.cc:374] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.677448 28655 raft_consensus.cc:729] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.677484 28655 consensus_queue.cc:260] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.677578 28655 ts_tablet_manager.cc:1398] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.677619 28655 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.677673 28656 tablet_bootstrap.cc:654] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.677862 28655 tablet_bootstrap.cc:654] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.677947 28664 tablet_bootstrap.cc:492] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.677985 28664 ts_tablet_manager.cc:1367] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.678105 28664 raft_consensus.cc:348] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.677095 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dabf28ec9f14b5998910b185852b22c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.678154 28664 raft_consensus.cc:374] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.678158 27583 raft_consensus.cc:3037] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.678138 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dabf28ec9f14b5998910b185852b22c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.678182 28664 raft_consensus.cc:729] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.678192 27722 raft_consensus.cc:3037] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.678211 28656 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.678218 28664 consensus_queue.cc:260] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.678246 28656 ts_tablet_manager.cc:1367] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.678288 28664 ts_tablet_manager.cc:1398] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.678344 28664 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.678347 28656 raft_consensus.cc:348] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.678391 28656 raft_consensus.cc:374] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.678406 28656 raft_consensus.cc:729] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.678454 28656 consensus_queue.cc:260] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.678565 28656 ts_tablet_manager.cc:1398] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.678612 28656 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.678654 28664 tablet_bootstrap.cc:654] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.678731 27583 raft_consensus.cc:2454] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.678786 27722 raft_consensus.cc:2454] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.678869 27386 leader_election.cc:302] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.678934 28767 raft_consensus.cc:2781] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.678963 28767 raft_consensus.cc:686] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.679023 28656 tablet_bootstrap.cc:654] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.679006 28767 consensus_queue.cc:237] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.679522 28655 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.679558 28655 ts_tablet_manager.cc:1367] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.679687 28655 raft_consensus.cc:348] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.679738 28655 raft_consensus.cc:374] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.679755 28655 raft_consensus.cc:729] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.679785 28655 consensus_queue.cc:260] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.679862 28655 ts_tablet_manager.cc:1398] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.679906 28655 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.680123 27194 catalog_manager.cc:5484] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.680205 28655 tablet_bootstrap.cc:654] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.680646 28654 ts_tablet_manager.cc:1909] T 00462246db1a4a53ba9f9a27a86ef538 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.680946 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 00462246db1a4a53ba9f9a27a86ef538 (table default.Table_22e35ec738b34697afb7f6d08708a4b9 [id=d7c0681b5f2e48b98160d0620659ef37]) successfully deleted
I20240503 02:09:14.681099 28654 tablet_replica.cc:330] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.681181 28654 raft_consensus.cc:2229] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.681269 28664 tablet_bootstrap.cc:492] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.681268 28654 raft_consensus.cc:2258] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.681313 28664 ts_tablet_manager.cc:1367] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.681449 28664 raft_consensus.cc:348] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.681506 28664 raft_consensus.cc:374] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.681528 28664 raft_consensus.cc:729] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.681565 28664 consensus_queue.cc:260] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.681613 28654 ts_tablet_manager.cc:1875] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.681630 28655 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.681659 28664 ts_tablet_manager.cc:1398] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.681667 28655 ts_tablet_manager.cc:1367] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.681721 28664 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.681792 28655 raft_consensus.cc:348] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.681842 28655 raft_consensus.cc:374] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.681860 28655 raft_consensus.cc:729] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.681891 28655 consensus_queue.cc:260] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.681962 28655 ts_tablet_manager.cc:1398] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.682015 28664 tablet_bootstrap.cc:654] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.682512 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
W20240503 02:09:14.682586 27194 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 59299d27526640828eda00961363d0a6 consensus state has no leader
I20240503 02:09:14.683094 28654 ts_tablet_manager.cc:1888] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.683110 28664 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.683141 28664 ts_tablet_manager.cc:1367] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.683271 28654 log.cc:1196] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9d1fbd4e5618451fac90622fbcf1e9bb
I20240503 02:09:14.683256 28664 raft_consensus.cc:348] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.683303 28664 raft_consensus.cc:374] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.683326 28664 raft_consensus.cc:729] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.683331 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.683358 28664 consensus_queue.cc:260] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.683449 28664 ts_tablet_manager.cc:1398] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.683507 28664 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.683699 28654 ts_tablet_manager.cc:1909] T 9d1fbd4e5618451fac90622fbcf1e9bb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.683815 28664 tablet_bootstrap.cc:654] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.683959 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9d1fbd4e5618451fac90622fbcf1e9bb (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:14.684165 28654 tablet_replica.cc:330] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.684186 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.684247 28654 raft_consensus.cc:2229] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.684285 28664 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.684305 28654 raft_consensus.cc:2258] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.684320 28664 ts_tablet_manager.cc:1367] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.684408 28654 ts_tablet_manager.cc:1875] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.684492 28664 raft_consensus.cc:348] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.684543 28664 raft_consensus.cc:374] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.684561 28664 raft_consensus.cc:729] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.684563 28656 tablet_bootstrap.cc:492] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.684603 28656 ts_tablet_manager.cc:1367] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:14.684595 28664 consensus_queue.cc:260] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.684665 28664 ts_tablet_manager.cc:1398] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.684710 28664 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.684715 28656 raft_consensus.cc:348] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.684759 28656 raft_consensus.cc:374] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.684778 28656 raft_consensus.cc:729] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.684811 28656 consensus_queue.cc:260] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.684895 28656 ts_tablet_manager.cc:1398] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.684939 28656 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.685030 28664 tablet_bootstrap.cc:654] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.685274 28656 tablet_bootstrap.cc:654] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.685633 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.685767 28654 ts_tablet_manager.cc:1888] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.685847 28654 log.cc:1196] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/da16da72302e49568ea8d2967f0d2e7a
I20240503 02:09:14.686054 28654 ts_tablet_manager.cc:1909] T da16da72302e49568ea8d2967f0d2e7a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.686332 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet da16da72302e49568ea8d2967f0d2e7a (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:14.686427 28654 tablet_replica.cc:330] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.686482 28654 raft_consensus.cc:2229] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.686517 28654 raft_consensus.cc:2258] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.686712 28654 ts_tablet_manager.cc:1875] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.686851 28656 tablet_bootstrap.cc:492] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.686898 28656 ts_tablet_manager.cc:1367] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.687009 28656 raft_consensus.cc:348] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.687070 28656 raft_consensus.cc:374] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.687065 28664 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.687099 28656 raft_consensus.cc:729] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.687122 28664 ts_tablet_manager.cc:1367] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.687137 28656 consensus_queue.cc:260] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.687223 28656 ts_tablet_manager.cc:1398] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.687234 28664 raft_consensus.cc:348] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.687271 28664 raft_consensus.cc:374] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.687286 28664 raft_consensus.cc:729] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.687291 28656 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.687332 28664 consensus_queue.cc:260] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.687637 28656 tablet_bootstrap.cc:654] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.688015 28664 ts_tablet_manager.cc:1398] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.688050 28654 ts_tablet_manager.cc:1888] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.688087 28664 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.688089 28654 log.cc:1196] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/6149f14895e34ea5994e3cbb94cf0bb4
I20240503 02:09:14.688395 28664 tablet_bootstrap.cc:654] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.688400 28654 ts_tablet_manager.cc:1909] T 6149f14895e34ea5994e3cbb94cf0bb4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.688634 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 6149f14895e34ea5994e3cbb94cf0bb4 (table default.Table_99f1ffda145649899d3aa39d2147bdd4 [id=d7fd1f8976ab427fb5bf25442148897e]) successfully deleted
I20240503 02:09:14.688726 28654 tablet_replica.cc:330] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.688798 28654 raft_consensus.cc:2229] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.688860 28654 raft_consensus.cc:2258] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.689028 28654 ts_tablet_manager.cc:1875] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.690089 28654 ts_tablet_manager.cc:1888] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.690136 28654 log.cc:1196] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/685ad701b1c143d4a77f6ec0c738c125
I20240503 02:09:14.690188 28656 tablet_bootstrap.cc:492] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.690238 28656 ts_tablet_manager.cc:1367] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.690279 28654 ts_tablet_manager.cc:1909] T 685ad701b1c143d4a77f6ec0c738c125 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.690326 28664 tablet_bootstrap.cc:492] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.690359 28664 ts_tablet_manager.cc:1367] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.690354 28656 raft_consensus.cc:348] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.690435 28656 raft_consensus.cc:374] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.690456 28656 raft_consensus.cc:729] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.690495 28656 consensus_queue.cc:260] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.690495 28664 raft_consensus.cc:348] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.690549 28664 raft_consensus.cc:374] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.690567 28664 raft_consensus.cc:729] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.690572 28656 ts_tablet_manager.cc:1398] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.690593 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 685ad701b1c143d4a77f6ec0c738c125 (table default.Table_390a24b35e2f40f982412479ea7ebaec [id=6668302ad6e745efa2a0464759730d66]) successfully deleted
I20240503 02:09:14.690604 28664 consensus_queue.cc:260] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.690631 28656 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.690692 28664 ts_tablet_manager.cc:1398] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.690833 28664 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.691037 28656 tablet_bootstrap.cc:654] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.691129 28664 tablet_bootstrap.cc:654] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.691895 28664 tablet_bootstrap.cc:492] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.692027 28664 ts_tablet_manager.cc:1367] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.692152 28664 raft_consensus.cc:348] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.692210 28664 raft_consensus.cc:374] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.692234 28664 raft_consensus.cc:729] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.692281 28664 consensus_queue.cc:260] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.692435 28664 ts_tablet_manager.cc:1398] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.692618 28656 tablet_bootstrap.cc:492] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.692664 28656 ts_tablet_manager.cc:1367] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.692788 28656 raft_consensus.cc:348] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.692852 28656 raft_consensus.cc:374] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.692871 28656 raft_consensus.cc:729] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.692919 28656 consensus_queue.cc:260] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.693007 28656 ts_tablet_manager.cc:1398] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.693078 28656 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.693406 28656 tablet_bootstrap.cc:654] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.693850 28656 tablet_bootstrap.cc:492] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.693889 28656 ts_tablet_manager.cc:1367] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.694007 28656 raft_consensus.cc:348] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.694056 28656 raft_consensus.cc:374] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.694072 28656 raft_consensus.cc:729] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.694104 28656 consensus_queue.cc:260] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.694178 28656 ts_tablet_manager.cc:1398] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.694222 28656 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.694540 28656 tablet_bootstrap.cc:654] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.694583 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.695179 28656 tablet_bootstrap.cc:492] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.695220 28656 ts_tablet_manager.cc:1367] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.695333 28656 raft_consensus.cc:348] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.695375 28656 raft_consensus.cc:374] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.695391 28656 raft_consensus.cc:729] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.695430 28656 consensus_queue.cc:260] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.695533 28656 ts_tablet_manager.cc:1398] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.695637 28656 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.695901 28656 tablet_bootstrap.cc:654] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.696482 28656 tablet_bootstrap.cc:492] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.696527 28656 ts_tablet_manager.cc:1367] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.696628 28656 raft_consensus.cc:348] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.696688 28656 raft_consensus.cc:374] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.696707 28656 raft_consensus.cc:729] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.696730 28656 consensus_queue.cc:260] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.696790 28656 ts_tablet_manager.cc:1398] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.696841 28656 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.697131 28656 tablet_bootstrap.cc:654] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.697399 28507 raft_consensus.cc:482] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.697444 28507 raft_consensus.cc:504] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.697546 28507 leader_election.cc:288] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.697628 28656 tablet_bootstrap.cc:492] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.697700 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b092e2b36f4f98bb87b9f01d6d3c3b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.697757 27722 raft_consensus.cc:2454] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.697799 28656 ts_tablet_manager.cc:1367] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.697855 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b092e2b36f4f98bb87b9f01d6d3c3b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.697932 27449 raft_consensus.cc:2454] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.697958 28656 raft_consensus.cc:348] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.698029 28656 raft_consensus.cc:374] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.698045 28656 raft_consensus.cc:729] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.698083 28656 consensus_queue.cc:260] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.698150 27523 leader_election.cc:302] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.698180 28656 ts_tablet_manager.cc:1398] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.698239 28507 raft_consensus.cc:2781] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.698285 28507 raft_consensus.cc:482] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.698303 28507 raft_consensus.cc:3037] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.698907 28507 raft_consensus.cc:504] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.698948 28766 raft_consensus.cc:482] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.698992 28766 raft_consensus.cc:504] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.699020 28507 leader_election.cc:288] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.699111 28766 leader_election.cc:288] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.699174 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b092e2b36f4f98bb87b9f01d6d3c3b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.699205 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e4ce547683468a9cb0e1683b248508" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.699230 27449 raft_consensus.cc:3037] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.699249 27583 raft_consensus.cc:2454] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.699371 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b092e2b36f4f98bb87b9f01d6d3c3b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.699424 27721 raft_consensus.cc:3037] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.699766 27449 raft_consensus.cc:2454] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.699908 27721 raft_consensus.cc:2454] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.699982 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e4ce547683468a9cb0e1683b248508" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.700034 27722 raft_consensus.cc:2454] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.700140 27523 leader_election.cc:302] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.700160 27388 leader_election.cc:302] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.700227 28766 raft_consensus.cc:2781] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.700270 28766 raft_consensus.cc:482] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.700250 28507 raft_consensus.cc:2781] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.700289 28766 raft_consensus.cc:3037] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.700304 28507 raft_consensus.cc:686] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.700346 28507 consensus_queue.cc:237] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.700886 28766 raft_consensus.cc:504] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.700928 27194 catalog_manager.cc:5484] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:14.700999 28766 leader_election.cc:288] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.701102 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e4ce547683468a9cb0e1683b248508" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.701122 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e4ce547683468a9cb0e1683b248508" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.701154 27722 raft_consensus.cc:3037] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.701174 27583 raft_consensus.cc:3037] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.701699 27583 raft_consensus.cc:2454] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.701752 27722 raft_consensus.cc:2454] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.701838 27386 leader_election.cc:302] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.701905 28766 raft_consensus.cc:2781] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.701948 28766 raft_consensus.cc:686] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.701982 28766 consensus_queue.cc:237] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.702992 27848 raft_consensus.cc:482] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.703045 27848 raft_consensus.cc:504] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.703125 27848 leader_election.cc:288] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.703318 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c351b66df551435ea01295f1f8dabd25" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.703342 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c351b66df551435ea01295f1f8dabd25" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.703377 27722 raft_consensus.cc:2454] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.703401 27449 raft_consensus.cc:2454] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.703491 27523 leader_election.cc:302] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.703574 27848 raft_consensus.cc:2781] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.703603 27848 raft_consensus.cc:482] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.703620 27848 raft_consensus.cc:3037] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.703684 27194 catalog_manager.cc:5484] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.703748 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.704169 27848 raft_consensus.cc:504] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.704229 27853 raft_consensus.cc:482] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.704272 27848 leader_election.cc:288] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.704273 27853 raft_consensus.cc:504] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.704365 27853 leader_election.cc:288] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.704428 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c351b66df551435ea01295f1f8dabd25" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.704471 27722 raft_consensus.cc:3037] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.704592 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c351b66df551435ea01295f1f8dabd25" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.704592 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f4bf62a461048fea8dda930b82f464e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.704619 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f4bf62a461048fea8dda930b82f464e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.704648 27583 raft_consensus.cc:2454] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.704646 27449 raft_consensus.cc:3037] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.704674 27451 raft_consensus.cc:2454] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.704790 27656 leader_election.cc:302] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.704926 27853 raft_consensus.cc:2781] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.704957 27853 raft_consensus.cc:482] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.704974 27853 raft_consensus.cc:3037] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.705154 27449 raft_consensus.cc:2454] T c351b66df551435ea01295f1f8dabd25 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.705286 27524 leader_election.cc:302] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.705417 27848 raft_consensus.cc:2781] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.705462 27848 raft_consensus.cc:686] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.705495 27848 consensus_queue.cc:237] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.704977 27722 raft_consensus.cc:2454] T c351b66df551435ea01295f1f8dabd25 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.705750 27853 raft_consensus.cc:504] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.705847 27853 leader_election.cc:288] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.705978 27203 catalog_manager.cc:5484] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:14.706122 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f4bf62a461048fea8dda930b82f464e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.706176 27583 raft_consensus.cc:3037] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.706308 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f4bf62a461048fea8dda930b82f464e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.706353 27449 raft_consensus.cc:3037] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.706677 27583 raft_consensus.cc:2454] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.706799 27449 raft_consensus.cc:2454] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.706816 27656 leader_election.cc:302] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.706882 27853 raft_consensus.cc:2781] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.706918 27853 raft_consensus.cc:686] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.706951 27853 consensus_queue.cc:237] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.707531 28740 raft_consensus.cc:482] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.707573 28740 raft_consensus.cc:504] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.707669 28740 leader_election.cc:288] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.708045 27194 catalog_manager.cc:5484] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.708127 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.708230 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51af2f65424fb0a7d26e13e4cd72ce" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.708292 27449 raft_consensus.cc:2454] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.708541 27659 leader_election.cc:302] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.708648 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51af2f65424fb0a7d26e13e4cd72ce" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.708714 27583 raft_consensus.cc:2454] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.708793 28740 raft_consensus.cc:2781] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.708837 28740 raft_consensus.cc:482] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.708865 28740 raft_consensus.cc:3037] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.709406 28740 raft_consensus.cc:504] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.709506 28740 leader_election.cc:288] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.709697 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51af2f65424fb0a7d26e13e4cd72ce" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.709753 27583 raft_consensus.cc:3037] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.709744 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc51af2f65424fb0a7d26e13e4cd72ce" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.709801 27449 raft_consensus.cc:3037] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.710119 27583 raft_consensus.cc:2454] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.710157 27449 raft_consensus.cc:2454] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.710264 27656 leader_election.cc:302] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.710350 28740 raft_consensus.cc:2781] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.710397 28740 raft_consensus.cc:686] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.710429 28740 consensus_queue.cc:237] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.710966 27194 catalog_manager.cc:5484] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.729883 28740 raft_consensus.cc:482] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.729928 28740 raft_consensus.cc:504] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.730023 28740 leader_election.cc:288] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.730129 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59299d27526640828eda00961363d0a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.730158 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59299d27526640828eda00961363d0a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.730206 27449 raft_consensus.cc:2454] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.730208 27583 raft_consensus.cc:2454] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.730324 27656 leader_election.cc:302] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.730443 28740 raft_consensus.cc:2781] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.730477 28740 raft_consensus.cc:482] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.730489 28740 raft_consensus.cc:3037] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.730897 28740 raft_consensus.cc:504] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.731019 28740 leader_election.cc:288] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.731132 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59299d27526640828eda00961363d0a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.731153 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59299d27526640828eda00961363d0a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.731204 27583 raft_consensus.cc:3037] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.731204 27449 raft_consensus.cc:3037] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.731612 27449 raft_consensus.cc:2454] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.731637 27583 raft_consensus.cc:2454] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.731763 27656 leader_election.cc:302] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.731885 28740 raft_consensus.cc:2781] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.731935 28740 raft_consensus.cc:686] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.731957 28740 consensus_queue.cc:237] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.732491 27194 catalog_manager.cc:5484] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.732571 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.741137 27449 raft_consensus.cc:1261] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.741140 27583 raft_consensus.cc:1261] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.741356 27853 consensus_queue.cc:1035] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.741452 28740 consensus_queue.cc:1035] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.742583 28673 mvcc.cc:205] Tried to move back new op lower bound from 7023420025818550272 to 7023420025679953920. Current Snapshot: MvccSnapshot[applied={T|T < 7023420025818550272}]
I20240503 02:09:14.742563 28706 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.742664 27203 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.742856 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.743099 28710 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.743456 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb]). Expected version 1 got 0
I20240503 02:09:14.744284 28698 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.745479 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_c24ee498ea4c4aaea463867f33b84d71 [id=b70cacd384524dd890671729a21e4fcb] alter complete (version 1)
I20240503 02:09:14.747916 27583 raft_consensus.cc:1261] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.747938 27722 raft_consensus.cc:1261] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.748126 28767 consensus_queue.cc:1035] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.748210 28766 consensus_queue.cc:1035] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.748993 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.749104 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.749531 28710 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.749544 28693 raft_consensus.cc:482] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.749593 28693 raft_consensus.cc:504] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.749701 28693 leader_election.cc:288] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.749814 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffa71e4ead479b801922f8a385be4e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.749869 27449 raft_consensus.cc:2454] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.750003 27659 leader_election.cc:302] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.750187 28698 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.750208 28693 raft_consensus.cc:2781] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.750236 28693 raft_consensus.cc:482] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.750252 28693 raft_consensus.cc:3037] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.750814 28693 raft_consensus.cc:504] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.750936 28693 leader_election.cc:288] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.751080 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.751191 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffa71e4ead479b801922f8a385be4e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.751245 27449 raft_consensus.cc:3037] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.751557 27848 raft_consensus.cc:482] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.751593 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffa71e4ead479b801922f8a385be4e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.751650 27587 raft_consensus.cc:3037] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.751770 27449 raft_consensus.cc:2454] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.752038 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_ecb958f7920a44ff9aebead671143001 [id=5f0720291c48410aa78697c47dd2831b]). Expected version 1 got 0
I20240503 02:09:14.752069 27659 leader_election.cc:302] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.751606 27848 raft_consensus.cc:504] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.752326 27848 leader_election.cc:288] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.752210 27587 raft_consensus.cc:2454] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.752676 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba4724c10854e468681870a4edda54b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.752735 27449 raft_consensus.cc:2454] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.752756 28706 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.752846 27524 leader_election.cc:302] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.752210 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffa71e4ead479b801922f8a385be4e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.752903 27583 raft_consensus.cc:2362] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:14.752995 27848 raft_consensus.cc:2781] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.753029 27848 raft_consensus.cc:482] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.753046 27848 raft_consensus.cc:3037] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.753064 28693 raft_consensus.cc:2781] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.753106 28693 raft_consensus.cc:686] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.753136 28693 consensus_queue.cc:237] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.753571 27848 raft_consensus.cc:504] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.753657 27848 leader_election.cc:288] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.753783 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba4724c10854e468681870a4edda54b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.753834 27449 raft_consensus.cc:3037] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.754293 27449 raft_consensus.cc:2454] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.754602 27524 leader_election.cc:302] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.754724 27848 raft_consensus.cc:2781] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.754763 27848 raft_consensus.cc:686] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.754797 27848 consensus_queue.cc:237] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.755328 27194 catalog_manager.cc:5484] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.755695 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba4724c10854e468681870a4edda54b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.755762 27721 raft_consensus.cc:3037] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.756150 27721 raft_consensus.cc:2454] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.756529 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba4724c10854e468681870a4edda54b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.756588 27722 raft_consensus.cc:2362] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:14.756911 27203 catalog_manager.cc:5484] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.786706 28507 raft_consensus.cc:482] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.786762 28507 raft_consensus.cc:504] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.786865 28507 leader_election.cc:288] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.786988 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d002acb8924f4d2194ed1e5f462ebfbc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.787000 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d002acb8924f4d2194ed1e5f462ebfbc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.787061 27449 raft_consensus.cc:2454] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.787061 27722 raft_consensus.cc:2454] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.787192 27524 leader_election.cc:302] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.787292 28507 raft_consensus.cc:2781] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.787338 28507 raft_consensus.cc:482] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.787355 28507 raft_consensus.cc:3037] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.787900 28507 raft_consensus.cc:504] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.788014 28507 leader_election.cc:288] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:14.788125 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d002acb8924f4d2194ed1e5f462ebfbc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.788179 27449 raft_consensus.cc:3037] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.788165 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d002acb8924f4d2194ed1e5f462ebfbc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.788218 27722 raft_consensus.cc:3037] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.788594 27449 raft_consensus.cc:2454] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.788743 27524 leader_election.cc:302] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.788741 27722 raft_consensus.cc:2454] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.788874 28507 raft_consensus.cc:2781] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.788942 28507 raft_consensus.cc:686] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.788985 28507 consensus_queue.cc:237] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.789526 27203 catalog_manager.cc:5484] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:14.792340 27203 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_efc4628674244e0a892ce7a48f15c83b" } modify_external_catalogs: true
I20240503 02:09:14.792404 27203 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_efc4628674244e0a892ce7a48f15c83b" } modify_external_catalogs: true
I20240503 02:09:14.793269 27203 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 0afe93270c134207a6845c1e49de1b4d
I20240503 02:09:14.793495 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 0afe93270c134207a6845c1e49de1b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.793571 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 0afe93270c134207a6845c1e49de1b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.793591 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 0afe93270c134207a6845c1e49de1b4d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.793658 28661 tablet_replica.cc:330] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.793661 28654 tablet_replica.cc:330] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.793705 28661 raft_consensus.cc:2229] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.793709 28654 raft_consensus.cc:2229] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.793740 28654 raft_consensus.cc:2258] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.793740 28661 raft_consensus.cc:2258] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.793797 28653 tablet_replica.cc:330] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.793852 28653 raft_consensus.cc:2229] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.793885 28653 raft_consensus.cc:2258] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.793888 28654 ts_tablet_manager.cc:1875] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.793934 28661 ts_tablet_manager.cc:1875] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.794062 28653 ts_tablet_manager.cc:1875] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.795012 28654 ts_tablet_manager.cc:1888] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.795054 28654 log.cc:1196] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0afe93270c134207a6845c1e49de1b4d
I20240503 02:09:14.795173 28653 ts_tablet_manager.cc:1888] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.795181 28661 ts_tablet_manager.cc:1888] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.795224 28661 log.cc:1196] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0afe93270c134207a6845c1e49de1b4d
I20240503 02:09:14.795224 28653 log.cc:1196] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0afe93270c134207a6845c1e49de1b4d
I20240503 02:09:14.795238 28654 ts_tablet_manager.cc:1909] T 0afe93270c134207a6845c1e49de1b4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.795372 28661 ts_tablet_manager.cc:1909] T 0afe93270c134207a6845c1e49de1b4d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.795372 28653 ts_tablet_manager.cc:1909] T 0afe93270c134207a6845c1e49de1b4d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.795578 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0afe93270c134207a6845c1e49de1b4d (table default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]) successfully deleted
I20240503 02:09:14.795626 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0afe93270c134207a6845c1e49de1b4d (table default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]) successfully deleted
I20240503 02:09:14.795663 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0afe93270c134207a6845c1e49de1b4d (table default.Table_efc4628674244e0a892ce7a48f15c83b [id=9356a04d7cc04d5ca3af5167d53410d9]) successfully deleted
I20240503 02:09:14.797247 27583 raft_consensus.cc:1261] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.797253 27449 raft_consensus.cc:1261] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.797469 28774 consensus_queue.cc:1035] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.797574 28693 consensus_queue.cc:1035] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.798151 27203 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.798553 27203 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 59299d27526640828eda00961363d0a6 (table default.Table_dd793faec53545858dfe59dce7a5066c [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 1 got 0
I20240503 02:09:14.798983 28706 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.799381 28710 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.799839 28698 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.800772 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet be7a66cc87834e739e958a2caf73de0c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.801698 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2170e94d17d44288964830bfd2f564e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.801811 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2170e94d17d44288964830bfd2f564e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.802935 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ff3d116e74aa4177b6f975650f4140c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.803041 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff3d116e74aa4177b6f975650f4140c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.803140 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_27a8698301354d42818e5744644ede84" } modify_external_catalogs: true
I20240503 02:09:14.803200 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_27a8698301354d42818e5744644ede84" } modify_external_catalogs: true
I20240503 02:09:14.803174 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5aa41bc4839d474fb29422980029dfa0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.803248 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5aa41bc4839d474fb29422980029dfa0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.803386 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a950b75e1ac54761af776af105ba5185"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.803467 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a950b75e1ac54761af776af105ba5185 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.803608 27203 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1fc11241cf4f4a228e9bc8d96114976a" } modify_external_catalogs: true
I20240503 02:09:14.803664 27203 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1fc11241cf4f4a228e9bc8d96114976a" } modify_external_catalogs: true
I20240503 02:09:14.803997 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet a571596f97cd4fcbb9247faaed98bf7c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.803979 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 5443787e1d464baf93549b49571f5687 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.805117 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 11c4e5ba1b274030b922c2234b0af7f3
I20240503 02:09:14.805346 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.805346 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.805433 27203 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 13b092e2b36f4f98bb87b9f01d6d3c3b
I20240503 02:09:14.805585 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 5443787e1d464baf93549b49571f5687
I20240503 02:09:14.805655 28661 tablet_replica.cc:330] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.805681 28653 tablet_replica.cc:330] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.805727 28661 raft_consensus.cc:2229] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.805728 28653 raft_consensus.cc:2229] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.805755 28661 raft_consensus.cc:2258] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.805775 28653 pending_rounds.cc:62] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:14.805792 28653 pending_rounds.cc:68] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025298083840 op_type: NO_OP noop_request { }
I20240503 02:09:14.805822 28653 raft_consensus.cc:2866] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.805841 28653 raft_consensus.cc:2258] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.805908 27422 tablet_service.cc:1467] Processing CreateTablet for tablet d3b844df0b674e038fa1fd7993033b30 (DEFAULT_TABLE table=default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.805986 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b844df0b674e038fa1fd7993033b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.806077 28661 ts_tablet_manager.cc:1875] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.806109 27698 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.806109 27701 tablet_service.cc:1467] Processing CreateTablet for tablet d3b844df0b674e038fa1fd7993033b30 (DEFAULT_TABLE table=default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.806277 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b844df0b674e038fa1fd7993033b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.807071 28655 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.807144 28767 raft_consensus.cc:482] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.807173 27421 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.807190 28767 raft_consensus.cc:504] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.807235 28664 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.807284 28767 leader_election.cc:288] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.807468 28655 tablet_bootstrap.cc:654] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.807548 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93b184492cf412d8e449e8c0527ea61" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.807579 28664 tablet_bootstrap.cc:654] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.807607 27722 raft_consensus.cc:2454] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.807860 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_72dc709d551446539881510584f3087c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.807917 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dbc2c33b084947c7854b5676739b95df" } new_table_name: "default.Table_a7e843c85f994c37b794b47e4adbdf70" modify_external_catalogs: true
I20240503 02:09:14.807992 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93b184492cf412d8e449e8c0527ea61" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.808058 27583 raft_consensus.cc:2454] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.808061 27199 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_059540465a8f493596bbda1239ae26b5" } new_table_name: "default.Table_0186eb69dc5645ce8e873d8cb0b95100" modify_external_catalogs: true
I20240503 02:09:14.808202 27386 leader_election.cc:302] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.808256 28664 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.808292 28664 ts_tablet_manager.cc:1367] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.808339 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet 5443787e1d464baf93549b49571f5687 with delete_type TABLET_DATA_DELETED (replaced by tablet ed866637e1fa4b9fbcee3bbd93fd785d) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.808363 28767 raft_consensus.cc:2781] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.808394 28767 raft_consensus.cc:482] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.808326 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b092e2b36f4f98bb87b9f01d6d3c3b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.808409 28767 raft_consensus.cc:3037] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.808436 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c4e5ba1b274030b922c2234b0af7f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.808432 28664 raft_consensus.cc:348] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.808318 27562 tablet_service.cc:1467] Processing CreateTablet for tablet d3b844df0b674e038fa1fd7993033b30 (DEFAULT_TABLE table=default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.808481 28664 raft_consensus.cc:374] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.808499 28664 raft_consensus.cc:729] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.808533 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3b844df0b674e038fa1fd7993033b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.808544 28664 consensus_queue.cc:260] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.808640 28664 ts_tablet_manager.cc:1398] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.808805 28655 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.808838 28655 ts_tablet_manager.cc:1367] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.808972 28767 raft_consensus.cc:504] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.808987 28655 raft_consensus.cc:348] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.809026 28655 raft_consensus.cc:374] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.809043 28655 raft_consensus.cc:729] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.809069 28767 leader_election.cc:288] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.809185 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93b184492cf412d8e449e8c0527ea61" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.809231 27583 raft_consensus.cc:3037] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.809557 28656 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.809689 27583 raft_consensus.cc:2454] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.809816 28656 tablet_bootstrap.cc:654] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.809075 28655 consensus_queue.cc:260] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.809788 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8f683e52b53441c48e52712f9271d74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.809897 28655 ts_tablet_manager.cc:1398] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.809967 27386 leader_election.cc:302] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.810032 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93b184492cf412d8e449e8c0527ea61" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.810084 27722 raft_consensus.cc:3037] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.810146 28767 raft_consensus.cc:2781] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.810184 28767 raft_consensus.cc:686] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.810215 28767 consensus_queue.cc:237] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.810286 28654 tablet_replica.cc:330] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.810343 28654 raft_consensus.cc:2229] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.810376 28654 raft_consensus.cc:2258] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.810614 27722 raft_consensus.cc:2454] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.810621 28654 ts_tablet_manager.cc:1875] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.810901 28656 tablet_bootstrap.cc:492] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.810936 28656 ts_tablet_manager.cc:1367] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.811059 28656 raft_consensus.cc:348] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.811107 28656 raft_consensus.cc:374] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.811123 28656 raft_consensus.cc:729] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.811164 28656 consensus_queue.cc:260] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.811249 28656 ts_tablet_manager.cc:1398] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.811730 28654 ts_tablet_manager.cc:1888] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:14.811769 28654 log.cc:1196] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/5443787e1d464baf93549b49571f5687
I20240503 02:09:14.811939 28654 ts_tablet_manager.cc:1909] T 5443787e1d464baf93549b49571f5687 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.812170 28654 tablet_replica.cc:330] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.812222 28654 raft_consensus.cc:2229] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.812260 28654 pending_rounds.cc:62] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:14.812275 28654 pending_rounds.cc:68] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025652899840 op_type: NO_OP noop_request { }
I20240503 02:09:14.812304 28654 raft_consensus.cc:2866] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.812325 28654 raft_consensus.cc:2258] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.812351 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.812476 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_acc60625100f46ca9a4b6b8ee71f3a4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:14.812719 28654 ts_tablet_manager.cc:1875] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.812803 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 5443787e1d464baf93549b49571f5687 deleted and replaced by tablet ed866637e1fa4b9fbcee3bbd93fd785d
I20240503 02:09:14.812919 27421 tablet_service.cc:1514] Processing DeleteTablet for tablet 5443787e1d464baf93549b49571f5687 with delete_type TABLET_DATA_DELETED (replaced by tablet ed866637e1fa4b9fbcee3bbd93fd785d) from {username='slave'} at 127.0.0.1:34416
W20240503 02:09:14.813004 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72dc709d551446539881510584f3087c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:14.813020 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f683e52b53441c48e52712f9271d74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:14.813113 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:14.813171 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acc60625100f46ca9a4b6b8ee71f3a4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.813205 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 5443787e1d464baf93549b49571f5687 with delete_type TABLET_DATA_DELETED (replaced by tablet ed866637e1fa4b9fbcee3bbd93fd785d) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.813903 28654 ts_tablet_manager.cc:1888] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.813946 28654 log.cc:1196] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/13b092e2b36f4f98bb87b9f01d6d3c3b
I20240503 02:09:14.814116 28654 ts_tablet_manager.cc:1909] T 13b092e2b36f4f98bb87b9f01d6d3c3b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.814585 28654 tablet_replica.cc:330] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.814700 28654 raft_consensus.cc:2229] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.814736 28654 raft_consensus.cc:2258] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.814949 28654 ts_tablet_manager.cc:1875] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.815393 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_91792da8cc4444c1b2910bf6dda7b4fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.815505 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91792da8cc4444c1b2910bf6dda7b4fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.816193 28710 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.816274 28654 ts_tablet_manager.cc:1888] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.816314 28654 log.cc:1196] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/11c4e5ba1b274030b922c2234b0af7f3
I20240503 02:09:14.816375 28698 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.816452 28654 ts_tablet_manager.cc:1909] T 11c4e5ba1b274030b922c2234b0af7f3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.818426 28706 tablet.cc:1722] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.818658 28710 tablet.cc:1722] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.819414 28698 tablet.cc:1722] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.820031 28694 tablet.cc:1722] T 56b36a3207ff43c185993f0e9ea457fa P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:14.821175 27421 tablet_service.cc:1467] Processing CreateTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (DEFAULT_TABLE table=default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.821264 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.821316 27423 tablet_service.cc:1467] Processing CreateTablet for tablet 2cbd420b867b4144985103d498b1ea73 (DEFAULT_TABLE table=default.Table_ff3d116e74aa4177b6f975650f4140c1 [id=ad081ee0810b483e873abf1fd1abf746]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.821386 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b (DEFAULT_TABLE table=default.Table_5aa41bc4839d474fb29422980029dfa0 [id=2921ee2db01a428eb4718fb9c08038bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.821430 27423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cbd420b867b4144985103d498b1ea73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.821659 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe7dc7197554c6ab1604d275eb67e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.822098 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 5443787e1d464baf93549b49571f5687 (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]) successfully deleted
I20240503 02:09:14.822150 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (table default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]) successfully deleted
I20240503 02:09:14.822176 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 11c4e5ba1b274030b922c2234b0af7f3 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.822341 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 2cbd420b867b4144985103d498b1ea73 (DEFAULT_TABLE table=default.Table_ff3d116e74aa4177b6f975650f4140c1 [id=ad081ee0810b483e873abf1fd1abf746]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.822422 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cbd420b867b4144985103d498b1ea73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.822507 28655 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.822664 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 2cbd420b867b4144985103d498b1ea73 (DEFAULT_TABLE table=default.Table_ff3d116e74aa4177b6f975650f4140c1 [id=ad081ee0810b483e873abf1fd1abf746]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.822712 27698 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b (DEFAULT_TABLE table=default.Table_5aa41bc4839d474fb29422980029dfa0 [id=2921ee2db01a428eb4718fb9c08038bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.822813 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (DEFAULT_TABLE table=default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.822824 28655 tablet_bootstrap.cc:654] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.823328 28655 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.823367 28655 ts_tablet_manager.cc:1367] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.823462 28656 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.823486 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b (DEFAULT_TABLE table=default.Table_5aa41bc4839d474fb29422980029dfa0 [id=2921ee2db01a428eb4718fb9c08038bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.823496 28655 raft_consensus.cc:348] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.823557 28655 raft_consensus.cc:374] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.823570 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe7dc7197554c6ab1604d275eb67e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.823576 28655 raft_consensus.cc:729] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.823622 28655 consensus_queue.cc:260] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.823706 28655 ts_tablet_manager.cc:1398] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.823727 28656 tablet_bootstrap.cc:654] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.823750 28655 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.824071 28655 tablet_bootstrap.cc:654] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.824235 28656 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.824263 28656 ts_tablet_manager.cc:1367] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.824366 28656 raft_consensus.cc:348] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.824410 28656 raft_consensus.cc:374] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.824421 28656 raft_consensus.cc:729] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.824452 28656 consensus_queue.cc:260] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.824554 28656 ts_tablet_manager.cc:1398] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.824625 27199 master_service.cc:946] ReplaceTablet: received request to replace tablet d75acbf10cd547e6925caca235db8f46 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.824729 28656 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.824838 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d1dccb2cfe654cce8410d7503dbf8985" } new_table_name: "default.Table_d895ab85cb064b40bdf6cebed80debbe" modify_external_catalogs: true
I20240503 02:09:14.824960 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet be7a66cc87834e739e958a2caf73de0c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:14.825060 28656 tablet_bootstrap.cc:654] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.825237 27203 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ef7ad145af794a2fab93caf50bce1418"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:14.825341 27203 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef7ad145af794a2fab93caf50bce1418 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:14.825655 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (DEFAULT_TABLE table=default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.825738 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.825810 28656 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.825848 28656 ts_tablet_manager.cc:1367] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.825985 28656 raft_consensus.cc:348] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.826031 28656 raft_consensus.cc:374] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.826048 28656 raft_consensus.cc:729] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.826084 28656 consensus_queue.cc:260] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.826197 28656 ts_tablet_manager.cc:1398] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:14.826625 28656 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.826781 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5aa41bc4839d474fb29422980029dfa0" } new_table_name: "default.Table_1193de5d35994a1cbc3bd480ac42bd2f" modify_external_catalogs: true
I20240503 02:09:14.827126 28661 ts_tablet_manager.cc:1888] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.827168 28661 log.cc:1196] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/11c4e5ba1b274030b922c2234b0af7f3
I20240503 02:09:14.827319 28661 ts_tablet_manager.cc:1909] T 11c4e5ba1b274030b922c2234b0af7f3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.827533 28661 tablet_replica.cc:330] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.827567 28656 tablet_bootstrap.cc:654] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.827581 28661 raft_consensus.cc:2229] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.827610 28661 raft_consensus.cc:2258] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.827811 27199 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d75acbf10cd547e6925caca235db8f46
I20240503 02:09:14.827877 27199 catalog_manager.cc:6554] ReplaceTablet: tablet d75acbf10cd547e6925caca235db8f46 deleted and replaced by tablet 3858a2107d6f434aaad8156a4f617115
I20240503 02:09:14.828001 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe7dc7197554c6ab1604d275eb67e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.828382 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cbd420b867b4144985103d498b1ea73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.828475 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.828601 27422 tablet_service.cc:1467] Processing CreateTablet for tablet b381283603ff4e84a6a399658044a071 (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.828683 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b381283603ff4e84a6a399658044a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.828709 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 09079fec9eae482ca5fccbcb69b5e61d (DEFAULT_TABLE table=default.Table_72dc709d551446539881510584f3087c [id=fd555a4517b94ba1842b9270634f7c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.828794 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09079fec9eae482ca5fccbcb69b5e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.828882 27423 tablet_service.cc:1514] Processing DeleteTablet for tablet d75acbf10cd547e6925caca235db8f46 with delete_type TABLET_DATA_DELETED (replaced by tablet 3858a2107d6f434aaad8156a4f617115) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.829391 27432 tablet_service.cc:1467] Processing CreateTablet for tablet ed866637e1fa4b9fbcee3bbd93fd785d (DEFAULT_TABLE table=default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.828681 27421 tablet_service.cc:1467] Processing CreateTablet for tablet 36bb12209a3a42f092c724ba8373d79b (DEFAULT_TABLE table=default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.829468 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed866637e1fa4b9fbcee3bbd93fd785d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.829528 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36bb12209a3a42f092c724ba8373d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.829557 28664 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.829870 28664 tablet_bootstrap.cc:654] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.830554 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 63e82b82ee7649d88867dbb9eef8a9de (DEFAULT_TABLE table=default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e [id=4908b7d914c64d328883e411ea95a2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.830633 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e82b82ee7649d88867dbb9eef8a9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.830854 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 11c4e5ba1b274030b922c2234b0af7f3 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.831071 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 09079fec9eae482ca5fccbcb69b5e61d (DEFAULT_TABLE table=default.Table_72dc709d551446539881510584f3087c [id=fd555a4517b94ba1842b9270634f7c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.831101 27699 tablet_service.cc:1467] Processing CreateTablet for tablet ed866637e1fa4b9fbcee3bbd93fd785d (DEFAULT_TABLE table=default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.831169 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed866637e1fa4b9fbcee3bbd93fd785d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.831079 27695 tablet_service.cc:1467] Processing CreateTablet for tablet 36bb12209a3a42f092c724ba8373d79b (DEFAULT_TABLE table=default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.831321 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36bb12209a3a42f092c724ba8373d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.831418 27698 tablet_service.cc:1467] Processing CreateTablet for tablet b381283603ff4e84a6a399658044a071 (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.831506 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b381283603ff4e84a6a399658044a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.832243 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 63e82b82ee7649d88867dbb9eef8a9de (DEFAULT_TABLE table=default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e [id=4908b7d914c64d328883e411ea95a2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.832322 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09079fec9eae482ca5fccbcb69b5e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.832480 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e82b82ee7649d88867dbb9eef8a9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.832648 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet d75acbf10cd547e6925caca235db8f46 with delete_type TABLET_DATA_DELETED (replaced by tablet 3858a2107d6f434aaad8156a4f617115) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.833019 27563 tablet_service.cc:1467] Processing CreateTablet for tablet ed866637e1fa4b9fbcee3bbd93fd785d (DEFAULT_TABLE table=default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.833015 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e23e16bec54548cbbbb515d698ffb45e" } modify_external_catalogs: true
I20240503 02:09:14.833070 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e23e16bec54548cbbbb515d698ffb45e" } modify_external_catalogs: true
I20240503 02:09:14.833102 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed866637e1fa4b9fbcee3bbd93fd785d. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:14.833230 27198 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9fe7dc7197554c6ab1604d275eb67e8b consensus state has no leader
I20240503 02:09:14.833660 28664 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.833698 28664 ts_tablet_manager.cc:1367] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:14.833832 28664 raft_consensus.cc:348] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.833886 28664 raft_consensus.cc:374] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.833902 28664 raft_consensus.cc:729] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.833936 28664 consensus_queue.cc:260] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.834009 28664 ts_tablet_manager.cc:1398] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.834049 28664 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.834182 27552 tablet_service.cc:1467] Processing CreateTablet for tablet 63e82b82ee7649d88867dbb9eef8a9de (DEFAULT_TABLE table=default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e [id=4908b7d914c64d328883e411ea95a2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.834264 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63e82b82ee7649d88867dbb9eef8a9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.834370 28664 tablet_bootstrap.cc:654] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.834697 27853 raft_consensus.cc:482] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.834735 27853 raft_consensus.cc:504] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.834829 27853 leader_election.cc:288] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.834954 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a571596f97cd4fcbb9247faaed98bf7c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.835008 28656 tablet_bootstrap.cc:492] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.835023 27449 raft_consensus.cc:2454] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.835042 28656 ts_tablet_manager.cc:1367] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:14.835140 27659 leader_election.cc:302] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.835143 27199 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_90367233df3b4c4680069a5fa47b4624" } modify_external_catalogs: true
I20240503 02:09:14.835160 28656 raft_consensus.cc:348] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.835194 27199 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_90367233df3b4c4680069a5fa47b4624" } modify_external_catalogs: true
I20240503 02:09:14.835204 28656 raft_consensus.cc:374] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.835222 28656 raft_consensus.cc:729] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.835261 28656 consensus_queue.cc:260] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.835321 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 09079fec9eae482ca5fccbcb69b5e61d (DEFAULT_TABLE table=default.Table_72dc709d551446539881510584f3087c [id=fd555a4517b94ba1842b9270634f7c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.835343 28656 ts_tablet_manager.cc:1398] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.835394 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09079fec9eae482ca5fccbcb69b5e61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.835415 27853 raft_consensus.cc:2781] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.835444 27853 raft_consensus.cc:482] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.835461 27853 raft_consensus.cc:3037] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.835471 28740 raft_consensus.cc:482] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.835510 28740 raft_consensus.cc:504] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.835578 28740 leader_election.cc:288] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.835855 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "503bd37c6def464496f6dd4489fca009" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.835911 27449 raft_consensus.cc:2454] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.835393 28656 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.836036 27853 raft_consensus.cc:504] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.836130 27853 leader_election.cc:288] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.836041 27659 leader_election.cc:302] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.836218 27853 raft_consensus.cc:2781] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.836246 27853 raft_consensus.cc:482] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.836261 27853 raft_consensus.cc:3037] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.836304 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a571596f97cd4fcbb9247faaed98bf7c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.836349 27449 raft_consensus.cc:3037] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.836354 28656 tablet_bootstrap.cc:654] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.836798 27853 raft_consensus.cc:504] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.836833 27449 raft_consensus.cc:2454] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.836863 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a571596f97cd4fcbb9247faaed98bf7c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.836900 27853 leader_election.cc:288] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.836920 27582 raft_consensus.cc:3037] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.836941 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "503bd37c6def464496f6dd4489fca009" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.836983 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a571596f97cd4fcbb9247faaed98bf7c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.837016 28710 tablet.cc:1722] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.836992 27587 raft_consensus.cc:2454] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.837142 27659 leader_election.cc:302] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.837455 28664 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.837461 28698 tablet.cc:1722] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.837489 28664 ts_tablet_manager.cc:1367] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:14.837560 27582 raft_consensus.cc:2454] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.837606 28664 raft_consensus.cc:348] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.837654 28664 raft_consensus.cc:374] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.837671 28664 raft_consensus.cc:729] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.837692 27853 raft_consensus.cc:2781] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.837658 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ff3d116e74aa4177b6f975650f4140c1" } new_table_name: "default.Table_ba19958baec64dc5988857e73a111865" modify_external_catalogs: true
I20240503 02:09:14.837706 28664 consensus_queue.cc:260] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.837728 27853 raft_consensus.cc:686] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.837761 27853 consensus_queue.cc:237] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.837782 28664 ts_tablet_manager.cc:1398] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.837833 28664 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.838133 28664 tablet_bootstrap.cc:654] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.838157 27565 tablet_service.cc:1467] Processing CreateTablet for tablet b381283603ff4e84a6a399658044a071 (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.838240 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b381283603ff4e84a6a399658044a071. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.838531 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "503bd37c6def464496f6dd4489fca009" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.838583 27449 raft_consensus.cc:3037] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.839057 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "503bd37c6def464496f6dd4489fca009" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.839107 27582 raft_consensus.cc:3037] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.839105 27449 raft_consensus.cc:2454] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.839515 27582 raft_consensus.cc:2454] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.839577 27659 leader_election.cc:302] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.839661 28693 raft_consensus.cc:2781] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.839700 28693 raft_consensus.cc:686] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.839730 28693 consensus_queue.cc:237] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.839740 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 36bb12209a3a42f092c724ba8373d79b (DEFAULT_TABLE table=default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.839816 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36bb12209a3a42f092c724ba8373d79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.840232 28664 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.840270 28664 ts_tablet_manager.cc:1367] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.840389 28664 raft_consensus.cc:348] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.840436 28664 raft_consensus.cc:374] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.840452 28664 raft_consensus.cc:729] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.840497 28664 consensus_queue.cc:260] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.840577 28664 ts_tablet_manager.cc:1398] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.840620 28664 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.840919 28664 tablet_bootstrap.cc:654] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.841475 28664 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.841516 28664 ts_tablet_manager.cc:1367] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.841645 28664 raft_consensus.cc:348] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.841692 28664 raft_consensus.cc:374] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.841710 28664 raft_consensus.cc:729] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.841745 28664 consensus_queue.cc:260] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.841833 28664 ts_tablet_manager.cc:1398] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.841876 28664 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.842304 28664 tablet_bootstrap.cc:654] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.842672 28653 ts_tablet_manager.cc:1875] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.843358 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2e99d982dddd497988d2d7220b96aefa
I20240503 02:09:14.843581 27199 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1e419af3f80e40b8a1acdfb71550ee91
I20240503 02:09:14.843675 27424 tablet_service.cc:1467] Processing CreateTablet for tablet f6022124cd954fcabb93d301a64837a8 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.843835 28653 ts_tablet_manager.cc:1888] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.843894 28653 log.cc:1196] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/11c4e5ba1b274030b922c2234b0af7f3
I20240503 02:09:14.843688 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 3858a2107d6f434aaad8156a4f617115 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.844065 28653 ts_tablet_manager.cc:1909] T 11c4e5ba1b274030b922c2234b0af7f3 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.843858 27429 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99d982dddd497988d2d7220b96aefa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.844179 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.844341 28664 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.844388 28664 ts_tablet_manager.cc:1367] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:14.844394 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 11c4e5ba1b274030b922c2234b0af7f3 (table default.Table_27a8698301354d42818e5744644ede84 [id=8c5af7bc1da7468d97ccf0b6bc92c831]) successfully deleted
I20240503 02:09:14.843834 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6022124cd954fcabb93d301a64837a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.844491 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3858a2107d6f434aaad8156a4f617115. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.844508 28664 raft_consensus.cc:348] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.844558 28664 raft_consensus.cc:374] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.844552 28653 tablet_replica.cc:330] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.844576 28664 raft_consensus.cc:729] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.844601 28653 raft_consensus.cc:2229] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.844606 28664 consensus_queue.cc:260] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.844630 28653 raft_consensus.cc:2258] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.844677 28664 ts_tablet_manager.cc:1398] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.844732 28664 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.844782 28653 ts_tablet_manager.cc:1875] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.845022 28664 tablet_bootstrap.cc:654] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.845484 28409 raft_consensus.cc:482] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.845525 28409 raft_consensus.cc:504] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.845613 28409 leader_election.cc:288] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
W20240503 02:09:14.845829 27202 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2cbd420b867b4144985103d498b1ea73 consensus state has no leader
I20240503 02:09:14.846007 27552 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.846076 28654 tablet_replica.cc:330] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.846055 27700 tablet_service.cc:1467] Processing CreateTablet for tablet f6022124cd954fcabb93d301a64837a8 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.846117 28654 raft_consensus.cc:2229] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.846101 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.846140 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6022124cd954fcabb93d301a64837a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.846154 27721 raft_consensus.cc:2454] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.846168 28654 raft_consensus.cc:2258] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:14.846174 27202 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:14.846225 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:14.846249 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:14.846251 27388 leader_election.cc:302] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.846349 28654 ts_tablet_manager.cc:1875] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.846060 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 3858a2107d6f434aaad8156a4f617115 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.846386 28409 raft_consensus.cc:2781] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.846414 28409 raft_consensus.cc:482] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.846427 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3858a2107d6f434aaad8156a4f617115. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.846431 28409 raft_consensus.cc:3037] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.846987 28409 raft_consensus.cc:504] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.847100 28409 leader_election.cc:288] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.847332 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.847386 27721 raft_consensus.cc:3037] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.847457 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e419af3f80e40b8a1acdfb71550ee91 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.847577 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 3858a2107d6f434aaad8156a4f617115 (DEFAULT_TABLE table=default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.847647 28653 ts_tablet_manager.cc:1888] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.847693 28653 log.cc:1196] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/13b092e2b36f4f98bb87b9f01d6d3c3b
I20240503 02:09:14.847945 28653 ts_tablet_manager.cc:1909] T 13b092e2b36f4f98bb87b9f01d6d3c3b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.848006 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99d982dddd497988d2d7220b96aefa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.847765 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3858a2107d6f434aaad8156a4f617115. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.848155 28653 tablet_replica.cc:330] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.848028 28654 ts_tablet_manager.cc:1888] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.848168 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.848206 28653 raft_consensus.cc:2229] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.848212 28654 log.cc:1196] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1e419af3f80e40b8a1acdfb71550ee91
I20240503 02:09:14.848235 28653 raft_consensus.cc:2258] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.848299 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (table default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]) successfully deleted
I20240503 02:09:14.848412 28654 ts_tablet_manager.cc:1909] T 1e419af3f80e40b8a1acdfb71550ee91 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.848490 28653 ts_tablet_manager.cc:1875] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.848215 27587 raft_consensus.cc:3037] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.849092 27587 raft_consensus.cc:2454] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.849225 27386 leader_election.cc:302] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.849251 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644]) successfully deleted
I20240503 02:09:14.849294 28409 raft_consensus.cc:2781] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.849349 28409 raft_consensus.cc:686] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.849393 28409 consensus_queue.cc:237] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.848073 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc72a7c4997486280a2fdcb3bc07c81" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.849753 27582 raft_consensus.cc:2362] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1. Re-sending same reply.
I20240503 02:09:14.850025 27721 raft_consensus.cc:2454] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.850737 28653 ts_tablet_manager.cc:1888] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:14.850790 28653 log.cc:1196] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/5443787e1d464baf93549b49571f5687
I20240503 02:09:14.851002 28653 ts_tablet_manager.cc:1909] T 5443787e1d464baf93549b49571f5687 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.851123 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a571596f97cd4fcbb9247faaed98bf7c
I20240503 02:09:14.851193 28653 tablet_replica.cc:330] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.851198 27194 catalog_manager.cc:6554] ReplaceTablet: tablet a571596f97cd4fcbb9247faaed98bf7c deleted and replaced by tablet 02a45a0a0f0a44099a10ec036aaf3acf
I20240503 02:09:14.851243 28653 raft_consensus.cc:2229] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.851284 28653 pending_rounds.cc:62] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:14.851280 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 5443787e1d464baf93549b49571f5687 (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]) successfully deleted
I20240503 02:09:14.851301 28653 pending_rounds.cc:68] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025425629184 op_type: NO_OP noop_request { }
I20240503 02:09:14.851332 28653 raft_consensus.cc:2866] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.851351 28653 raft_consensus.cc:2258] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.851620 28653 ts_tablet_manager.cc:1875] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.851737 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet a571596f97cd4fcbb9247faaed98bf7c with delete_type TABLET_DATA_DELETED (replaced by tablet 02a45a0a0f0a44099a10ec036aaf3acf) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.851744 27567 tablet_service.cc:1514] Processing DeleteTablet for tablet a571596f97cd4fcbb9247faaed98bf7c with delete_type TABLET_DATA_DELETED (replaced by tablet 02a45a0a0f0a44099a10ec036aaf3acf) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.851822 28654 tablet_replica.cc:330] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.851866 28654 raft_consensus.cc:2229] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.851891 28654 raft_consensus.cc:2258] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.851922 28661 ts_tablet_manager.cc:1875] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.851994 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet a571596f97cd4fcbb9247faaed98bf7c with delete_type TABLET_DATA_DELETED (replaced by tablet 02a45a0a0f0a44099a10ec036aaf3acf) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.852073 28654 ts_tablet_manager.cc:1875] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.852659 28653 ts_tablet_manager.cc:1888] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.852710 28653 log.cc:1196] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d75acbf10cd547e6925caca235db8f46
I20240503 02:09:14.852861 28653 ts_tablet_manager.cc:1909] T d75acbf10cd547e6925caca235db8f46 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.853123 28654 ts_tablet_manager.cc:1888] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.853135 28661 ts_tablet_manager.cc:1888] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.853168 28664 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.853214 28664 ts_tablet_manager.cc:1367] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:14.853281 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d75acbf10cd547e6925caca235db8f46 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.853174 28654 log.cc:1196] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a571596f97cd4fcbb9247faaed98bf7c
I20240503 02:09:14.853354 28664 raft_consensus.cc:348] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.853425 28664 raft_consensus.cc:374] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.853430 28653 tablet_replica.cc:330] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.853446 28664 raft_consensus.cc:729] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.853475 28653 raft_consensus.cc:2229] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.853505 28653 raft_consensus.cc:2258] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.853479 28664 consensus_queue.cc:260] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.853590 28664 ts_tablet_manager.cc:1398] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:14.853648 28664 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.853797 28653 ts_tablet_manager.cc:1875] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.853850 28654 ts_tablet_manager.cc:1909] T a571596f97cd4fcbb9247faaed98bf7c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.853967 28664 tablet_bootstrap.cc:654] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.854127 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a571596f97cd4fcbb9247faaed98bf7c (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:14.854480 28664 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.854523 28664 ts_tablet_manager.cc:1367] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.854638 28664 raft_consensus.cc:348] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.854687 28664 raft_consensus.cc:374] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.854704 28664 raft_consensus.cc:729] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.854739 28664 consensus_queue.cc:260] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.854830 28664 ts_tablet_manager.cc:1398] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.854876 28664 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.854914 28653 ts_tablet_manager.cc:1888] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.854959 28653 log.cc:1196] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2e99d982dddd497988d2d7220b96aefa
I20240503 02:09:14.855067 27567 tablet_service.cc:1467] Processing CreateTablet for tablet c583b44cecad4e93a50fddae3a0035b7 (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.855127 28653 ts_tablet_manager.cc:1909] T 2e99d982dddd497988d2d7220b96aefa P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.855142 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c583b44cecad4e93a50fddae3a0035b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.855222 28664 tablet_bootstrap.cc:654] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.855456 28653 tablet_replica.cc:330] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.855497 28653 raft_consensus.cc:2229] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.855525 28653 raft_consensus.cc:2258] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.856091 28653 ts_tablet_manager.cc:1875] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.856353 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2e99d982dddd497988d2d7220b96aefa (table default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.856755 27422 tablet_service.cc:1467] Processing CreateTablet for tablet c583b44cecad4e93a50fddae3a0035b7 (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.856844 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c583b44cecad4e93a50fddae3a0035b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.857431 28653 ts_tablet_manager.cc:1888] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.857424 27196 catalog_manager.cc:5484] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.857590 28653 log.cc:1196] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1e419af3f80e40b8a1acdfb71550ee91
I20240503 02:09:14.857604 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_0186eb69dc5645ce8e873d8cb0b95100 [id=04a56dd0fac64f1a845e5603e39fc641] alter complete (version 1)
I20240503 02:09:14.857734 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.857810 28653 ts_tablet_manager.cc:1909] T 1e419af3f80e40b8a1acdfb71550ee91 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.857825 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a45a0a0f0a44099a10ec036aaf3acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.858031 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.858052 27697 tablet_service.cc:1467] Processing CreateTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.858099 28740 raft_consensus.cc:482] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.858114 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a45a0a0f0a44099a10ec036aaf3acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.858139 27697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a45a0a0f0a44099a10ec036aaf3acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.858131 28740 raft_consensus.cc:504] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.858229 28740 leader_election.cc:288] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.858600 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b844df0b674e038fa1fd7993033b30" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.858656 27449 raft_consensus.cc:2454] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.858763 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b844df0b674e038fa1fd7993033b30" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.858798 27659 leader_election.cc:302] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.858817 27582 raft_consensus.cc:2454] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.858860 28740 raft_consensus.cc:2781] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.858882 28740 raft_consensus.cc:482] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.858898 28740 raft_consensus.cc:3037] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.858052 27700 tablet_service.cc:1467] Processing CreateTablet for tablet c583b44cecad4e93a50fddae3a0035b7 (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.859149 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644]) successfully deleted
I20240503 02:09:14.859169 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c583b44cecad4e93a50fddae3a0035b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.859244 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99d982dddd497988d2d7220b96aefa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.859311 28654 tablet_replica.cc:330] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.859352 28654 raft_consensus.cc:2229] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.859385 28654 raft_consensus.cc:2258] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.859447 27560 tablet_service.cc:1467] Processing CreateTablet for tablet f6022124cd954fcabb93d301a64837a8 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:14.859444 28740 raft_consensus.cc:504] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.859514 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6022124cd954fcabb93d301a64837a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:14.859548 28740 leader_election.cc:288] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.859910 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b844df0b674e038fa1fd7993033b30" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.859970 27449 raft_consensus.cc:3037] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.860109 28653 tablet_replica.cc:330] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.860162 28653 raft_consensus.cc:2229] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.860184 28653 raft_consensus.cc:2258] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.860285 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02] alter complete (version 1)
I20240503 02:09:14.860370 28653 ts_tablet_manager.cc:1875] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.860440 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:14.860492 27449 raft_consensus.cc:2454] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.860517 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3b844df0b674e038fa1fd7993033b30" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.860556 27582 raft_consensus.cc:3037] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.860491 27201 catalog_manager.cc:5484] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.860589 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:14.860678 27659 leader_election.cc:302] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.860743 28740 raft_consensus.cc:2781] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.860787 28740 raft_consensus.cc:686] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.860826 28740 consensus_queue.cc:237] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.861061 27582 raft_consensus.cc:2454] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.861409 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet d75acbf10cd547e6925caca235db8f46 with delete_type TABLET_DATA_DELETED (replaced by tablet 3858a2107d6f434aaad8156a4f617115) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.853174 28661 log.cc:1196] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/13b092e2b36f4f98bb87b9f01d6d3c3b
I20240503 02:09:14.861562 28654 ts_tablet_manager.cc:1875] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.861655 28661 ts_tablet_manager.cc:1909] T 13b092e2b36f4f98bb87b9f01d6d3c3b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.861882 28661 tablet_replica.cc:330] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.862093 28661 raft_consensus.cc:2229] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.862160 28661 raft_consensus.cc:2258] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.862224 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 13b092e2b36f4f98bb87b9f01d6d3c3b (table default.Table_1fc11241cf4f4a228e9bc8d96114976a [id=8a1705572b144c72ba3d600e6eed1d08]) successfully deleted
I20240503 02:09:14.862448 28661 ts_tablet_manager.cc:1875] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.862640 28654 ts_tablet_manager.cc:1888] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.862686 28654 log.cc:1196] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2e99d982dddd497988d2d7220b96aefa
I20240503 02:09:14.862819 28656 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.862857 28656 ts_tablet_manager.cc:1367] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.001s
I20240503 02:09:14.862859 28654 ts_tablet_manager.cc:1909] T 2e99d982dddd497988d2d7220b96aefa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.863160 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_a7e843c85f994c37b794b47e4adbdf70 [id=f62f02a2dfdd454e9ffdfe34656eaac3] alter complete (version 2)
I20240503 02:09:14.863202 28654 tablet_replica.cc:330] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.863251 28654 raft_consensus.cc:2229] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.863276 28654 raft_consensus.cc:2258] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.863008 28656 raft_consensus.cc:348] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:14.863345 27201 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet ed866637e1fa4b9fbcee3bbd93fd785d consensus state has no leader
I20240503 02:09:14.863368 28656 raft_consensus.cc:374] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.863384 28656 raft_consensus.cc:729] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.863420 28656 consensus_queue.cc:260] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.863497 28656 ts_tablet_manager.cc:1398] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.863487 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2e99d982dddd497988d2d7220b96aefa (table default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.863557 28656 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.863602 28655 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.863636 28655 ts_tablet_manager.cc:1367] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.000s
I20240503 02:09:14.863641 28661 ts_tablet_manager.cc:1888] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:14.863754 28655 raft_consensus.cc:348] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.863804 28655 raft_consensus.cc:374] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.863801 28661 log.cc:1196] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/5443787e1d464baf93549b49571f5687
I20240503 02:09:14.863824 28655 raft_consensus.cc:729] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.863859 28655 consensus_queue.cc:260] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.863865 28656 tablet_bootstrap.cc:654] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.863934 28655 ts_tablet_manager.cc:1398] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.864336 28655 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.864339 28661 ts_tablet_manager.cc:1909] T 5443787e1d464baf93549b49571f5687 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.864362 27201 catalog_manager.cc:5484] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.864578 28654 ts_tablet_manager.cc:1875] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.864660 28655 tablet_bootstrap.cc:654] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.865846 28654 ts_tablet_manager.cc:1888] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.865932 28654 log.cc:1196] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d75acbf10cd547e6925caca235db8f46
I20240503 02:09:14.866099 28654 ts_tablet_manager.cc:1909] T d75acbf10cd547e6925caca235db8f46 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.866295 28653 ts_tablet_manager.cc:1888] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.866348 28653 log.cc:1196] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a571596f97cd4fcbb9247faaed98bf7c
I20240503 02:09:14.866482 28653 ts_tablet_manager.cc:1909] T a571596f97cd4fcbb9247faaed98bf7c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.866910 28656 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.866952 28656 ts_tablet_manager.cc:1367] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:14.867012 28664 tablet_bootstrap.cc:492] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.867050 28664 ts_tablet_manager.cc:1367] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:14.867070 28656 raft_consensus.cc:348] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.867115 28656 raft_consensus.cc:374] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.867130 28656 raft_consensus.cc:729] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.867154 28656 consensus_queue.cc:260] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.867172 28664 raft_consensus.cc:348] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.867208 28661 tablet_replica.cc:330] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.867223 28664 raft_consensus.cc:374] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.867223 28656 ts_tablet_manager.cc:1398] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.867242 28664 raft_consensus.cc:729] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.867250 28661 raft_consensus.cc:2229] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.867264 28656 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.867277 28664 consensus_queue.cc:260] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.867357 28664 ts_tablet_manager.cc:1398] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.867419 28664 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.867581 28656 tablet_bootstrap.cc:654] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.867751 28664 tablet_bootstrap.cc:654] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.867846 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 5443787e1d464baf93549b49571f5687 (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]) successfully deleted
I20240503 02:09:14.867276 28661 raft_consensus.cc:2258] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.868211 28661 ts_tablet_manager.cc:1875] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.869275 28661 ts_tablet_manager.cc:1888] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.869397 28664 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.869448 28664 ts_tablet_manager.cc:1367] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:14.869447 28661 log.cc:1196] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d75acbf10cd547e6925caca235db8f46
I20240503 02:09:14.869534 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d75acbf10cd547e6925caca235db8f46 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.869603 28664 raft_consensus.cc:348] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.869645 28661 ts_tablet_manager.cc:1909] T d75acbf10cd547e6925caca235db8f46 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.869663 28706 tablet.cc:1722] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.869655 28664 raft_consensus.cc:374] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.869715 28664 raft_consensus.cc:729] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.869748 28664 consensus_queue.cc:260] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.869850 28664 ts_tablet_manager.cc:1398] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.869904 28664 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.869972 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a571596f97cd4fcbb9247faaed98bf7c (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:14.870011 28661 tablet_replica.cc:330] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.870061 28661 raft_consensus.cc:2229] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.870087 28661 raft_consensus.cc:2258] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.870141 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:14.870223 28664 tablet_bootstrap.cc:654] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.870327 28661 ts_tablet_manager.cc:1875] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.870616 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d75acbf10cd547e6925caca235db8f46 (table default.Table_7310e9433672464690114ff02b5c773f [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:14.872239 27196 catalog_manager.cc:5484] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.872452 28661 ts_tablet_manager.cc:1888] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.872545 28661 log.cc:1196] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1e419af3f80e40b8a1acdfb71550ee91
I20240503 02:09:14.872565 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:14.872905 28661 ts_tablet_manager.cc:1909] T 1e419af3f80e40b8a1acdfb71550ee91 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.873157 28661 tablet_replica.cc:330] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.873198 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1e419af3f80e40b8a1acdfb71550ee91 (table default.Table_90367233df3b4c4680069a5fa47b4624 [id=3383f9af04da490a8cb6f949d3d92644]) successfully deleted
I20240503 02:09:14.873250 28661 raft_consensus.cc:2229] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.873611 28661 raft_consensus.cc:2258] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.873881 28661 ts_tablet_manager.cc:1875] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.873893 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:14.874814 28661 ts_tablet_manager.cc:1888] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:14.874858 28661 log.cc:1196] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2e99d982dddd497988d2d7220b96aefa
I20240503 02:09:14.874998 28661 ts_tablet_manager.cc:1909] T 2e99d982dddd497988d2d7220b96aefa P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.875165 28661 tablet_replica.cc:330] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.875257 28661 raft_consensus.cc:2229] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.875260 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2e99d982dddd497988d2d7220b96aefa (table default.Table_e23e16bec54548cbbbb515d698ffb45e [id=3e130a5d5d654901b4db32907b5f526f]) successfully deleted
I20240503 02:09:14.875304 28661 pending_rounds.cc:62] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:14.875340 28661 pending_rounds.cc:68] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420026215657472 op_type: NO_OP noop_request { }
I20240503 02:09:14.875385 28661 raft_consensus.cc:2866] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.875412 28661 raft_consensus.cc:2258] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.875638 28661 ts_tablet_manager.cc:1875] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.876513 28661 ts_tablet_manager.cc:1888] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.876554 28661 log.cc:1196] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a571596f97cd4fcbb9247faaed98bf7c
I20240503 02:09:14.876689 28661 ts_tablet_manager.cc:1909] T a571596f97cd4fcbb9247faaed98bf7c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.876945 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a571596f97cd4fcbb9247faaed98bf7c (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:14.877604 28656 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.877647 28656 ts_tablet_manager.cc:1367] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:14.877664 28655 tablet_bootstrap.cc:492] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.877705 28655 ts_tablet_manager.cc:1367] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:14.877714 28664 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.877745 28664 ts_tablet_manager.cc:1367] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:14.877763 28656 raft_consensus.cc:348] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.877808 28656 raft_consensus.cc:374] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.877823 28656 raft_consensus.cc:729] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.877854 28656 consensus_queue.cc:260] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.877840 28655 raft_consensus.cc:348] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.877902 28655 raft_consensus.cc:374] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.877866 28664 raft_consensus.cc:348] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.877920 28655 raft_consensus.cc:729] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.877935 28664 raft_consensus.cc:374] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.877944 28656 ts_tablet_manager.cc:1398] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.877954 28664 raft_consensus.cc:729] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.877959 28655 consensus_queue.cc:260] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.877985 28664 consensus_queue.cc:260] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.878017 28656 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.878038 28655 ts_tablet_manager.cc:1398] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.878054 28664 ts_tablet_manager.cc:1398] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.878103 28655 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.878109 28664 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.878329 28656 tablet_bootstrap.cc:654] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.878365 28664 tablet_bootstrap.cc:654] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.878388 28655 tablet_bootstrap.cc:654] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.878931 28693 raft_consensus.cc:482] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.878973 28693 raft_consensus.cc:504] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.879060 28693 leader_election.cc:288] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.879201 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.879254 27582 raft_consensus.cc:2454] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.879563 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.879770 27656 leader_election.cc:302] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.879900 28693 raft_consensus.cc:2781] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.879931 28693 raft_consensus.cc:482] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.879947 28693 raft_consensus.cc:3037] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.880445 28693 raft_consensus.cc:504] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.880537 28693 leader_election.cc:288] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
W20240503 02:09:14.880638 27659 leader_election.cc:341] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:14.880681 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.880771 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.880800 27848 raft_consensus.cc:482] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.880815 27582 raft_consensus.cc:3037] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.880857 27848 raft_consensus.cc:504] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:14.880892 27659 leader_election.cc:341] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:14.881000 27848 leader_election.cc:288] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.881120 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9448d14b53f74e7f8ccbc19bb07b3a6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.881165 27722 raft_consensus.cc:2454] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.881381 27523 leader_election.cc:302] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.881421 27582 raft_consensus.cc:2454] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.881443 27848 raft_consensus.cc:2781] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.881466 27848 raft_consensus.cc:482] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.881482 27848 raft_consensus.cc:3037] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.881561 27656 leader_election.cc:302] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:14.882076 27848 raft_consensus.cc:504] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.882128 28693 raft_consensus.cc:2781] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.882175 28693 raft_consensus.cc:686] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.882179 27848 leader_election.cc:288] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.882205 28693 consensus_queue.cc:237] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.882236 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9448d14b53f74e7f8ccbc19bb07b3a6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.882300 27449 raft_consensus.cc:2454] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.882488 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9448d14b53f74e7f8ccbc19bb07b3a6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.882546 27722 raft_consensus.cc:3037] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.882534 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9448d14b53f74e7f8ccbc19bb07b3a6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.882588 27449 raft_consensus.cc:3037] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.883122 27449 raft_consensus.cc:2454] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.883257 27524 leader_election.cc:302] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.883091 27722 raft_consensus.cc:2454] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.881287 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:14.883553 28655 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.883589 28655 ts_tablet_manager.cc:1367] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:14.883711 27848 raft_consensus.cc:2781] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.883706 28655 raft_consensus.cc:348] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.883755 28655 raft_consensus.cc:374] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.883754 27848 raft_consensus.cc:686] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.883775 28655 raft_consensus.cc:729] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.883729 27196 catalog_manager.cc:5484] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.883791 27848 consensus_queue.cc:237] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.883807 28655 consensus_queue.cc:260] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.883883 28655 ts_tablet_manager.cc:1398] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.883960 28655 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.884284 28655 tablet_bootstrap.cc:654] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.884414 27201 catalog_manager.cc:5484] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.885397 28655 tablet_bootstrap.cc:492] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.885444 28655 ts_tablet_manager.cc:1367] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.886107 28656 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.886106 28664 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.886147 28656 ts_tablet_manager.cc:1367] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:14.886147 28664 ts_tablet_manager.cc:1367] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:14.886240 28655 raft_consensus.cc:348] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.886301 28655 raft_consensus.cc:374] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.886279 28664 raft_consensus.cc:348] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.886279 28656 raft_consensus.cc:348] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.886322 28655 raft_consensus.cc:729] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.886329 28664 raft_consensus.cc:374] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.886329 28656 raft_consensus.cc:374] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.886348 28664 raft_consensus.cc:729] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.886348 28656 raft_consensus.cc:729] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.886345 28655 consensus_queue.cc:260] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.886385 28664 consensus_queue.cc:260] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.886389 28656 consensus_queue.cc:260] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.886479 28656 ts_tablet_manager.cc:1398] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.886476 28664 ts_tablet_manager.cc:1398] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.886543 28664 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:14.886543 28656 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.886833 28656 tablet_bootstrap.cc:654] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.887102 28655 ts_tablet_manager.cc:1398] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.887348 28655 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.887820 28664 tablet_bootstrap.cc:654] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.888119 28655 tablet_bootstrap.cc:654] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.888175 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:14.888924 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:14.889089 28655 tablet_bootstrap.cc:492] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.889138 28655 ts_tablet_manager.cc:1367] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:14.889286 28655 raft_consensus.cc:348] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.889364 28655 raft_consensus.cc:374] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.889431 28655 raft_consensus.cc:729] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.889482 28655 consensus_queue.cc:260] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.889777 28655 ts_tablet_manager.cc:1398] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.889902 28655 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.890246 28656 tablet_bootstrap.cc:492] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.890288 28656 ts_tablet_manager.cc:1367] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:14.890311 28664 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:14.890344 28664 ts_tablet_manager.cc:1367] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:14.890403 28656 raft_consensus.cc:348] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.890450 28656 raft_consensus.cc:374] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.890427 28664 raft_consensus.cc:348] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.890468 28656 raft_consensus.cc:729] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.890470 28664 raft_consensus.cc:374] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.890476 28655 tablet_bootstrap.cc:654] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.890487 28664 raft_consensus.cc:729] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.890515 28656 consensus_queue.cc:260] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.890522 28664 consensus_queue.cc:260] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.890594 28664 ts_tablet_manager.cc:1398] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.890591 28656 ts_tablet_manager.cc:1398] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.890661 28656 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.890962 28656 tablet_bootstrap.cc:654] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.891287 28655 tablet_bootstrap.cc:492] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.891325 28655 ts_tablet_manager.cc:1367] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.891445 28655 raft_consensus.cc:348] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.891494 28655 raft_consensus.cc:374] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.891510 28655 raft_consensus.cc:729] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.891542 28655 consensus_queue.cc:260] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.891631 28655 ts_tablet_manager.cc:1398] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.891697 28655 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.892016 28655 tablet_bootstrap.cc:654] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.892630 28655 tablet_bootstrap.cc:492] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.892676 28655 ts_tablet_manager.cc:1367] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.892788 28655 raft_consensus.cc:348] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.892832 28655 raft_consensus.cc:374] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.892853 28655 raft_consensus.cc:729] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.892886 28655 consensus_queue.cc:260] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.892923 28409 raft_consensus.cc:482] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.892969 28409 raft_consensus.cc:504] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.892993 28655 ts_tablet_manager.cc:1398] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.893040 28655 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.893049 28409 leader_election.cc:288] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.893316 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7dd4d111794cf2a1fc7b65acd55fe3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.893432 28655 tablet_bootstrap.cc:654] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.893440 27582 raft_consensus.cc:2454] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.893446 28656 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.893559 28656 ts_tablet_manager.cc:1367] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.893662 28656 raft_consensus.cc:348] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.893723 28656 raft_consensus.cc:374] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.893743 28656 raft_consensus.cc:729] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.893777 28656 consensus_queue.cc:260] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.893805 27386 leader_election.cc:302] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.893785 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7dd4d111794cf2a1fc7b65acd55fe3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.893842 27722 raft_consensus.cc:2454] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.893867 28656 ts_tablet_manager.cc:1398] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.893925 28409 raft_consensus.cc:2781] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.893952 28409 raft_consensus.cc:482] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.893968 28409 raft_consensus.cc:3037] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.894392 28656 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.894598 28409 raft_consensus.cc:504] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.894694 28409 leader_election.cc:288] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.894716 28656 tablet_bootstrap.cc:654] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.894856 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7dd4d111794cf2a1fc7b65acd55fe3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.894906 27582 raft_consensus.cc:3037] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.895314 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7dd4d111794cf2a1fc7b65acd55fe3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.895363 28656 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.895373 27722 raft_consensus.cc:3037] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.895401 28656 ts_tablet_manager.cc:1367] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.895438 27582 raft_consensus.cc:2454] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.895506 28656 raft_consensus.cc:348] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.895552 28656 raft_consensus.cc:374] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.895567 28656 raft_consensus.cc:729] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.895568 27386 leader_election.cc:302] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.895591 28656 consensus_queue.cc:260] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.895655 28409 raft_consensus.cc:2781] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.895679 28656 ts_tablet_manager.cc:1398] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.895687 28409 raft_consensus.cc:686] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.895725 28656 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.895720 28409 consensus_queue.cc:237] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.895946 27722 raft_consensus.cc:2454] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.896262 28656 tablet_bootstrap.cc:654] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.896474 28655 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.896512 28655 ts_tablet_manager.cc:1367] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:14.896608 28655 raft_consensus.cc:348] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.896660 28655 raft_consensus.cc:374] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.896677 28655 raft_consensus.cc:729] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.896708 28655 consensus_queue.cc:260] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.896685 27202 catalog_manager.cc:5484] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.896759 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.896783 28655 ts_tablet_manager.cc:1398] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.896817 28656 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.896827 28655 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.896852 28656 ts_tablet_manager.cc:1367] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:14.896960 28656 raft_consensus.cc:348] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.897008 28656 raft_consensus.cc:374] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.897027 28656 raft_consensus.cc:729] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.897066 28656 consensus_queue.cc:260] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.897145 28656 ts_tablet_manager.cc:1398] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.897146 28655 tablet_bootstrap.cc:654] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.897210 28656 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:14.898524 28655 tablet_bootstrap.cc:492] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.898555 28656 tablet_bootstrap.cc:654] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.898630 28655 ts_tablet_manager.cc:1367] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.898782 28655 raft_consensus.cc:348] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.898886 28655 raft_consensus.cc:374] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.898941 28655 raft_consensus.cc:729] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.899055 28656 tablet_bootstrap.cc:492] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:14.899111 28656 ts_tablet_manager.cc:1367] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:14.899219 28656 raft_consensus.cc:348] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.899264 28656 raft_consensus.cc:374] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.899279 28656 raft_consensus.cc:729] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.899320 28656 consensus_queue.cc:260] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.899397 28656 ts_tablet_manager.cc:1398] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.899035 28655 consensus_queue.cc:260] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.899616 28655 ts_tablet_manager.cc:1398] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.899760 28655 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.900141 28655 tablet_bootstrap.cc:654] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.900781 28655 tablet_bootstrap.cc:492] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.900821 28655 ts_tablet_manager.cc:1367] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:14.900990 28655 raft_consensus.cc:348] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.901037 28655 raft_consensus.cc:374] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.901049 28655 raft_consensus.cc:729] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.901080 28655 consensus_queue.cc:260] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.901298 28655 ts_tablet_manager.cc:1398] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.901437 28655 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:14.901970 28655 tablet_bootstrap.cc:654] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:14.902520 28655 tablet_bootstrap.cc:492] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:14.902555 28655 ts_tablet_manager.cc:1367] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:14.902659 28655 raft_consensus.cc:348] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.902704 28655 raft_consensus.cc:374] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:14.902722 28655 raft_consensus.cc:729] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:14.902765 28655 consensus_queue.cc:260] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.902833 28655 ts_tablet_manager.cc:1398] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:14.912724 28740 raft_consensus.cc:482] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.912772 28740 raft_consensus.cc:504] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.912870 28740 leader_election.cc:288] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.912986 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bb12209a3a42f092c724ba8373d79b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.913010 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bb12209a3a42f092c724ba8373d79b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.913044 27449 raft_consensus.cc:2454] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.913069 27582 raft_consensus.cc:2454] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.913146 27659 leader_election.cc:302] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.913228 28740 raft_consensus.cc:2781] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.913267 28740 raft_consensus.cc:482] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.913285 28740 raft_consensus.cc:3037] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.913698 28740 raft_consensus.cc:504] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.913810 28740 leader_election.cc:288] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.913931 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bb12209a3a42f092c724ba8373d79b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.913946 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bb12209a3a42f092c724ba8373d79b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.913996 27582 raft_consensus.cc:3037] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.913990 27449 raft_consensus.cc:3037] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.914382 27449 raft_consensus.cc:2454] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.914516 27582 raft_consensus.cc:2454] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.914533 27659 leader_election.cc:302] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.914608 28740 raft_consensus.cc:2781] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.914675 28740 raft_consensus.cc:686] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.914716 28740 consensus_queue.cc:237] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.915253 27202 catalog_manager.cc:5484] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.923185 28409 raft_consensus.cc:482] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.923229 28409 raft_consensus.cc:504] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.923312 28409 leader_election.cc:288] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.923434 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6022124cd954fcabb93d301a64837a8" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.923434 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6022124cd954fcabb93d301a64837a8" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.923485 27582 raft_consensus.cc:2454] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.923485 27722 raft_consensus.cc:2454] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:14.923602 27386 leader_election.cc:302] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.923689 28409 raft_consensus.cc:2781] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.923727 28409 raft_consensus.cc:482] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.923739 28409 raft_consensus.cc:3037] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.924142 28409 raft_consensus.cc:504] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.924233 28409 leader_election.cc:288] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.924348 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6022124cd954fcabb93d301a64837a8" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.924362 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6022124cd954fcabb93d301a64837a8" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.924420 27722 raft_consensus.cc:3037] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.924422 27582 raft_consensus.cc:3037] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.924877 27582 raft_consensus.cc:2454] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.924899 27722 raft_consensus.cc:2454] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:14.925029 27388 leader_election.cc:302] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.925120 28409 raft_consensus.cc:2781] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.925175 28409 raft_consensus.cc:686] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:14.925216 28409 consensus_queue.cc:237] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.925825 27202 catalog_manager.cc:5484] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.931905 28507 raft_consensus.cc:482] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.931953 28507 raft_consensus.cc:504] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.932063 28507 leader_election.cc:288] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.932165 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a45a0a0f0a44099a10ec036aaf3acf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.932181 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a45a0a0f0a44099a10ec036aaf3acf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.932233 27722 raft_consensus.cc:2454] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.932233 27449 raft_consensus.cc:2454] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:14.932363 27523 leader_election.cc:302] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.932440 28507 raft_consensus.cc:2781] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.932476 28507 raft_consensus.cc:482] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.932513 28507 raft_consensus.cc:3037] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.932929 28507 raft_consensus.cc:504] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.933038 28507 leader_election.cc:288] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:14.933141 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a45a0a0f0a44099a10ec036aaf3acf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.933188 27449 raft_consensus.cc:3037] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.933393 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a45a0a0f0a44099a10ec036aaf3acf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:14.933446 27722 raft_consensus.cc:3037] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.933652 27449 raft_consensus.cc:2454] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.933820 27524 leader_election.cc:302] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:14.933857 27722 raft_consensus.cc:2454] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:14.933888 28507 raft_consensus.cc:2781] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.933931 28507 raft_consensus.cc:686] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:14.933964 28507 consensus_queue.cc:237] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:14.934630 27202 catalog_manager.cc:5484] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.936666 28740 raft_consensus.cc:482] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.936712 28740 raft_consensus.cc:504] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.936787 28740 leader_election.cc:288] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.936895 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6285b6ea40c84ba9a16c6d1c39b2f3d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.936935 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6285b6ea40c84ba9a16c6d1c39b2f3d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.936964 27449 raft_consensus.cc:2454] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.936983 27582 raft_consensus.cc:2454] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.937085 27659 leader_election.cc:302] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.937173 28740 raft_consensus.cc:2781] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.937203 28740 raft_consensus.cc:482] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.937215 28740 raft_consensus.cc:3037] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.937649 28740 raft_consensus.cc:504] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.937741 28740 leader_election.cc:288] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.937861 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6285b6ea40c84ba9a16c6d1c39b2f3d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.937878 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6285b6ea40c84ba9a16c6d1c39b2f3d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.937920 27582 raft_consensus.cc:3037] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.937922 27449 raft_consensus.cc:3037] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.938408 27582 raft_consensus.cc:2454] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.938408 27449 raft_consensus.cc:2454] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.938581 27659 leader_election.cc:302] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.938650 28740 raft_consensus.cc:2781] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.938711 28740 raft_consensus.cc:686] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.938750 28740 consensus_queue.cc:237] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.939517 27202 catalog_manager.cc:5484] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:14.952710 28409 raft_consensus.cc:482] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.952760 28409 raft_consensus.cc:504] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.952852 28409 leader_election.cc:288] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.952976 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.952989 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09079fec9eae482ca5fccbcb69b5e61d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:14.953048 27582 raft_consensus.cc:2379] T 09079fec9eae482ca5fccbcb69b5e61d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:14.953161 27386 leader_election.cc:302] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:14.953243 28409 raft_consensus.cc:3037] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.953701 28409 raft_consensus.cc:2726] T 09079fec9eae482ca5fccbcb69b5e61d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:14.965423 27722 raft_consensus.cc:1261] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.965430 27582 raft_consensus.cc:1261] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:14.965587 28767 consensus_queue.cc:1035] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.965668 28409 consensus_queue.cc:1035] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.966423 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.966711 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.966951 28692 mvcc.cc:205] Tried to move back new op lower bound from 7023420026737197056 to 7023420026453155840. Current Snapshot: MvccSnapshot[applied={T|T < 7023420026737197056}]
I20240503 02:09:14.967144 28710 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.967314 28698 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.967372 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2c7dd4d111794cf2a1fc7b65acd55fe3 (table default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00]). Expected version 1 got 0
I20240503 02:09:14.967870 28706 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:14.969357 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_21661264ffe84bba9777c3f8f8d15678 [id=352a12ca13d94899afacb8e670b99d00] alter complete (version 1)
I20240503 02:09:14.979398 28740 consensus_queue.cc:1035] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.980907 28693 consensus_queue.cc:1035] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:14.993835 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_33dbbeaec27e4fa6909562489deb85c9" } modify_external_catalogs: true
I20240503 02:09:14.993896 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_33dbbeaec27e4fa6909562489deb85c9" } modify_external_catalogs: true
I20240503 02:09:14.994764 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1ba4724c10854e468681870a4edda54b
I20240503 02:09:14.994964 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba4724c10854e468681870a4edda54b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:14.994971 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba4724c10854e468681870a4edda54b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:14.995043 28661 tablet_replica.cc:330] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:14.995052 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba4724c10854e468681870a4edda54b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:14 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:14.995067 28654 tablet_replica.cc:330] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:14.995090 28661 raft_consensus.cc:2229] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.995116 28661 raft_consensus.cc:2258] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.995119 28654 raft_consensus.cc:2229] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:14.995136 28653 tablet_replica.cc:330] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:14.995167 28654 pending_rounds.cc:62] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:14.995184 28654 pending_rounds.cc:68] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025875927040 op_type: NO_OP noop_request { }
I20240503 02:09:14.995195 28653 raft_consensus.cc:2229] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:14.995213 28654 raft_consensus.cc:2866] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:14.995226 28653 raft_consensus.cc:2258] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.995231 28654 raft_consensus.cc:2258] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:14.995369 28653 ts_tablet_manager.cc:1875] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.995499 28661 ts_tablet_manager.cc:1875] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.995568 28654 ts_tablet_manager.cc:1875] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:14.996663 28661 ts_tablet_manager.cc:1888] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.996663 28653 ts_tablet_manager.cc:1888] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:14.996692 28654 ts_tablet_manager.cc:1888] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:14.996716 28661 log.cc:1196] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1ba4724c10854e468681870a4edda54b
I20240503 02:09:14.996722 28653 log.cc:1196] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1ba4724c10854e468681870a4edda54b
I20240503 02:09:14.996743 28654 log.cc:1196] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1ba4724c10854e468681870a4edda54b
I20240503 02:09:14.996874 28661 ts_tablet_manager.cc:1909] T 1ba4724c10854e468681870a4edda54b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:14.996874 28653 ts_tablet_manager.cc:1909] T 1ba4724c10854e468681870a4edda54b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:14.996951 28654 ts_tablet_manager.cc:1909] T 1ba4724c10854e468681870a4edda54b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:14.997228 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1ba4724c10854e468681870a4edda54b (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.997345 28693 raft_consensus.cc:482] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:14.997397 28693 raft_consensus.cc:504] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.997414 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1ba4724c10854e468681870a4edda54b (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.997493 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1ba4724c10854e468681870a4edda54b (table default.Table_33dbbeaec27e4fa6909562489deb85c9 [id=4f451132d8cb414dbf46a16b407359d9]) successfully deleted
I20240503 02:09:14.997499 28693 leader_election.cc:288] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.997610 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe7dc7197554c6ab1604d275eb67e8b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:14.997606 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe7dc7197554c6ab1604d275eb67e8b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:14.997666 27449 raft_consensus.cc:2454] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.997666 27582 raft_consensus.cc:2454] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:14.997778 27659 leader_election.cc:302] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.997859 28693 raft_consensus.cc:2781] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:14.997895 28693 raft_consensus.cc:482] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:14.997910 28693 raft_consensus.cc:3037] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.998302 28693 raft_consensus.cc:504] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:14.998414 28693 leader_election.cc:288] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:14.998695 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe7dc7197554c6ab1604d275eb67e8b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:14.998704 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe7dc7197554c6ab1604d275eb67e8b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:14.998756 27451 raft_consensus.cc:3037] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:14.998764 27582 raft_consensus.cc:3037] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:14.998806 27201 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:14.999277 27582 raft_consensus.cc:2454] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.999308 27451 raft_consensus.cc:2454] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:14.999433 27656 leader_election.cc:302] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:14.999538 28740 raft_consensus.cc:2781] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:14.999583 28740 raft_consensus.cc:686] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:14.999615 28740 consensus_queue.cc:237] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.000185 27201 catalog_manager.cc:5484] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.000291 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:15.002207 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 8cbf03ee6db446169cdd24e91c11bdcd from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.003087 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8cbf03ee6db446169cdd24e91c11bdcd
I20240503 02:09:15.003181 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 8cbf03ee6db446169cdd24e91c11bdcd deleted and replaced by tablet 739e17504abc42d0a54cd92b1f171fca
I20240503 02:09:15.003362 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd with delete_type TABLET_DATA_DELETED (replaced by tablet 739e17504abc42d0a54cd92b1f171fca) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.003453 28653 tablet_replica.cc:330] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.003475 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd with delete_type TABLET_DATA_DELETED (replaced by tablet 739e17504abc42d0a54cd92b1f171fca) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.003508 28653 raft_consensus.cc:2229] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.003542 28661 tablet_replica.cc:330] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.003548 28653 raft_consensus.cc:2258] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.003588 28661 raft_consensus.cc:2229] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.003635 28661 raft_consensus.cc:2258] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.003648 28653 ts_tablet_manager.cc:1875] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.003721 28661 ts_tablet_manager.cc:1875] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.004805 28661 ts_tablet_manager.cc:1888] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.004848 28661 log.cc:1196] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8cbf03ee6db446169cdd24e91c11bdcd
I20240503 02:09:15.004945 28653 ts_tablet_manager.cc:1888] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.004994 28653 log.cc:1196] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8cbf03ee6db446169cdd24e91c11bdcd
I20240503 02:09:15.005023 28661 ts_tablet_manager.cc:1909] T 8cbf03ee6db446169cdd24e91c11bdcd P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.005201 28653 ts_tablet_manager.cc:1909] T 8cbf03ee6db446169cdd24e91c11bdcd P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.005311 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cbf03ee6db446169cdd24e91c11bdcd with delete_type TABLET_DATA_DELETED (replaced by tablet 739e17504abc42d0a54cd92b1f171fca) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.005380 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.005486 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.005528 28654 tablet_replica.cc:330] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.005597 28654 raft_consensus.cc:2229] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.005636 28654 raft_consensus.cc:2258] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.005702 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 739e17504abc42d0a54cd92b1f171fca (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.005694 27700 tablet_service.cc:1467] Processing CreateTablet for tablet 739e17504abc42d0a54cd92b1f171fca (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.005738 28654 ts_tablet_manager.cc:1875] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.005785 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739e17504abc42d0a54cd92b1f171fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.005789 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739e17504abc42d0a54cd92b1f171fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.005879 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:15.006659 28664 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.006891 28656 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.006956 28654 ts_tablet_manager.cc:1888] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.006963 28664 tablet_bootstrap.cc:654] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.006994 28654 log.cc:1196] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8cbf03ee6db446169cdd24e91c11bdcd
I20240503 02:09:15.006963 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_59cd63322a1a4147a3929468a00f9fa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.007055 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59cd63322a1a4147a3929468a00f9fa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.007187 28654 ts_tablet_manager.cc:1909] T 8cbf03ee6db446169cdd24e91c11bdcd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.007243 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:15.007253 28656 tablet_bootstrap.cc:654] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.007571 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 739e17504abc42d0a54cd92b1f171fca (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.007674 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739e17504abc42d0a54cd92b1f171fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.007740 28656 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.007786 28656 ts_tablet_manager.cc:1367] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.007928 28656 raft_consensus.cc:348] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.007980 28656 raft_consensus.cc:374] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.007998 28656 raft_consensus.cc:729] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.008040 28656 consensus_queue.cc:260] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.008128 28656 ts_tablet_manager.cc:1398] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.008101 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0186eb69dc5645ce8e873d8cb0b95100" } modify_external_catalogs: true
I20240503 02:09:15.008164 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0186eb69dc5645ce8e873d8cb0b95100" } modify_external_catalogs: true
I20240503 02:09:15.008334 28664 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.008376 28664 ts_tablet_manager.cc:1367] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.008507 28664 raft_consensus.cc:348] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.008565 28664 raft_consensus.cc:374] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.008584 28664 raft_consensus.cc:729] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.008553 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5d89254081104ad3be52d93900293b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.008559 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a950b75e1ac54761af776af105ba5185" } modify_external_catalogs: true
I20240503 02:09:15.008618 28664 consensus_queue.cc:260] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.008644 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a950b75e1ac54761af776af105ba5185" } modify_external_catalogs: true
I20240503 02:09:15.008695 28664 ts_tablet_manager.cc:1398] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.008715 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_df5d4f768010413ab307211f72391619"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.008782 28655 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:15.008803 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df5d4f768010413ab307211f72391619 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.009097 28655 tablet_bootstrap.cc:654] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:15.008647 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d89254081104ad3be52d93900293b1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.009208 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8cbf03ee6db446169cdd24e91c11bdcd (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:15.009789 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
W20240503 02:09:15.009857 27195 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 739e17504abc42d0a54cd92b1f171fca consensus state has no leader
I20240503 02:09:15.009879 27203 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.010344 28655 tablet_bootstrap.cc:492] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.010398 28655 ts_tablet_manager.cc:1367] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.010525 28655 raft_consensus.cc:348] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.010573 28655 raft_consensus.cc:374] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.010589 28655 raft_consensus.cc:729] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.010623 28655 consensus_queue.cc:260] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.010710 28655 ts_tablet_manager.cc:1398] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.011025 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.011364 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.012063 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2
I20240503 02:09:15.012229 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:15.012331 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.012396 28661 tablet_replica.cc:330] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.012437 28661 raft_consensus.cc:2229] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.012480 28661 pending_rounds.cc:62] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.012497 28661 pending_rounds.cc:68] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420026629283840 op_type: NO_OP noop_request { }
I20240503 02:09:15.012526 28661 raft_consensus.cc:2866] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.012540 28661 raft_consensus.cc:2258] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.012557 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.012780 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.012821 28661 ts_tablet_manager.cc:1875] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.013010 27582 raft_consensus.cc:1261] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.013036 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.013110 27451 raft_consensus.cc:1261] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.013178 28693 consensus_queue.cc:1035] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.013335 28653 tablet_replica.cc:330] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.013396 28653 raft_consensus.cc:2229] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.013386 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.013435 28653 raft_consensus.cc:2258] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.013440 28693 consensus_queue.cc:1035] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.013604 28653 ts_tablet_manager.cc:1875] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.013919 28661 ts_tablet_manager.cc:1888] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.013962 28661 log.cc:1196] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/6285b6ea40c84ba9a16c6d1c39b2f3d2
I20240503 02:09:15.014145 28661 ts_tablet_manager.cc:1909] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.014490 28693 raft_consensus.cc:482] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.014532 28693 raft_consensus.cc:504] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.014622 28693 leader_election.cc:288] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.014667 28653 ts_tablet_manager.cc:1888] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.014709 28653 log.cc:1196] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:15.014700 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:15.014808 28654 tablet_replica.cc:330] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.014801 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739e17504abc42d0a54cd92b1f171fca" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.014854 27449 raft_consensus.cc:2454] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.014864 28654 raft_consensus.cc:2229] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.014878 28653 ts_tablet_manager.cc:1909] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.014899 28654 raft_consensus.cc:2258] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.015023 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739e17504abc42d0a54cd92b1f171fca" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.015074 27587 raft_consensus.cc:2454] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.015108 27659 leader_election.cc:302] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.015125 28654 ts_tablet_manager.cc:1875] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.015544 28693 raft_consensus.cc:2781] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.015583 28693 raft_consensus.cc:482] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.015600 28693 raft_consensus.cc:3037] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.016144 28693 raft_consensus.cc:504] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.016182 28654 ts_tablet_manager.cc:1888] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.016224 28654 log.cc:1196] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:15.016245 28693 leader_election.cc:288] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.016407 28654 ts_tablet_manager.cc:1909] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.016505 28706 tablet.cc:1722] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.016605 28710 tablet.cc:1722] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.016671 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739e17504abc42d0a54cd92b1f171fca" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.016717 27587 raft_consensus.cc:3037] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.016855 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (table default.Table_0186eb69dc5645ce8e873d8cb0b95100 [id=04a56dd0fac64f1a845e5603e39fc641]) successfully deleted
I20240503 02:09:15.017263 27587 raft_consensus.cc:2454] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.017297 27199 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:15.017402 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739e17504abc42d0a54cd92b1f171fca" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.017448 27451 raft_consensus.cc:3037] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.017480 27656 leader_election.cc:302] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.017483 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]). Expected version 1 got 0
I20240503 02:09:15.017547 28693 raft_consensus.cc:2781] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.017583 28693 raft_consensus.cc:686] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.017613 28693 consensus_queue.cc:237] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.017658 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.017732 28653 tablet_replica.cc:330] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.017782 28653 raft_consensus.cc:2229] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.017809 28653 raft_consensus.cc:2258] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.017892 28661 tablet_replica.cc:330] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.017943 28661 raft_consensus.cc:2229] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.018010 28661 raft_consensus.cc:2258] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.017993 27451 raft_consensus.cc:2454] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.018088 28653 ts_tablet_manager.cc:1875] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.018218 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (table default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]) successfully deleted
I20240503 02:09:15.018441 28661 ts_tablet_manager.cc:1875] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.019083 28653 ts_tablet_manager.cc:1888] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.019130 28653 log.cc:1196] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/6285b6ea40c84ba9a16c6d1c39b2f3d2
I20240503 02:09:15.019260 28653 ts_tablet_manager.cc:1909] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.019791 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (table default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]) successfully deleted
I20240503 02:09:15.020262 27199 catalog_manager.cc:5484] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.020355 27199 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.020677 28654 tablet_replica.cc:330] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.020725 28654 raft_consensus.cc:2229] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.020754 28654 raft_consensus.cc:2258] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.020807 28698 tablet.cc:1722] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.020922 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (table default.Table_0186eb69dc5645ce8e873d8cb0b95100 [id=04a56dd0fac64f1a845e5603e39fc641]) successfully deleted
I20240503 02:09:15.020826 28665 mvcc.cc:205] Tried to move back new op lower bound from 7023420026931556352 to 7023420026878640128. Current Snapshot: MvccSnapshot[applied={T|T < 7023420026931556352}]
I20240503 02:09:15.021620 28661 ts_tablet_manager.cc:1888] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.021673 28661 log.cc:1196] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3aae0d9cd2ec4a7cb86ef48ef02f8ee6
I20240503 02:09:15.021865 28661 ts_tablet_manager.cc:1909] T 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.022114 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3aae0d9cd2ec4a7cb86ef48ef02f8ee6 (table default.Table_0186eb69dc5645ce8e873d8cb0b95100 [id=04a56dd0fac64f1a845e5603e39fc641]) successfully deleted
I20240503 02:09:15.022976 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e76fc1dd5ef5469cbe8c9a673743fb0e" } new_table_name: "default.Table_679392a2422e43f88ca52faf229d218a" modify_external_catalogs: true
I20240503 02:09:15.023327 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c24ee498ea4c4aaea463867f33b84d71" } new_table_name: "default.Table_c6bdef5afa174922afb43e03000feebb" modify_external_catalogs: true
I20240503 02:09:15.023383 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf] alter complete (version 1)
I20240503 02:09:15.023842 27203 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_aded67e380fd48b987de136312959fd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.023952 27203 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aded67e380fd48b987de136312959fd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.024088 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_630051f741264c3a925573b248899ffd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.024154 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_630051f741264c3a925573b248899ffd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.024258 27199 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_37701508cc6c4db8b07876a048312845"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.024334 27199 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37701508cc6c4db8b07876a048312845 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.024631 28740 consensus_queue.cc:1035] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.024683 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_eec82c477df84e9dbd2b9874505d9e48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.024766 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eec82c477df84e9dbd2b9874505d9e48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.025133 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet 092ad3ae09fe47bbbe473643060c43d0 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.026338 28774 consensus_queue.cc:1035] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.026625 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_72b3b65538eb417192d7f431d42c12cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.026716 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72b3b65538eb417192d7f431d42c12cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.027662 28710 tablet.cc:1722] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.027812 28698 tablet.cc:1722] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.027964 28706 tablet.cc:1722] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.028563 28654 ts_tablet_manager.cc:1875] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.030215 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:15.030292 27194 catalog_manager.cc:6554] ReplaceTablet: tablet 092ad3ae09fe47bbbe473643060c43d0 deleted and replaced by tablet 78d80210ca3842fba4243bafc7498b2a
I20240503 02:09:15.030529 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 092ad3ae09fe47bbbe473643060c43d0 with delete_type TABLET_DATA_DELETED (replaced by tablet 78d80210ca3842fba4243bafc7498b2a) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.030903 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 092ad3ae09fe47bbbe473643060c43d0 with delete_type TABLET_DATA_DELETED (replaced by tablet 78d80210ca3842fba4243bafc7498b2a) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.030987 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet a93b184492cf412d8e449e8c0527ea61 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.031157 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 092ad3ae09fe47bbbe473643060c43d0 with delete_type TABLET_DATA_DELETED (replaced by tablet 78d80210ca3842fba4243bafc7498b2a) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.031224 28653 tablet_replica.cc:330] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.031265 28653 raft_consensus.cc:2229] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.031267 28704 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.031314 28653 raft_consensus.cc:2258] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.031409 28653 ts_tablet_manager.cc:1875] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.031812 28654 ts_tablet_manager.cc:1888] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.031845 28654 log.cc:1196] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/6285b6ea40c84ba9a16c6d1c39b2f3d2
I20240503 02:09:15.031955 28654 ts_tablet_manager.cc:1909] T 6285b6ea40c84ba9a16c6d1c39b2f3d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.032039 28661 tablet_replica.cc:330] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.032088 28661 raft_consensus.cc:2229] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.032120 28661 raft_consensus.cc:2258] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.032344 28661 ts_tablet_manager.cc:1875] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.032570 28710 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.032820 28654 tablet_replica.cc:330] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.032869 28654 raft_consensus.cc:2229] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.032904 28654 raft_consensus.cc:2258] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.032894 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 6285b6ea40c84ba9a16c6d1c39b2f3d2 (table default.Table_a950b75e1ac54761af776af105ba5185 [id=14c3e5df6eeb49e3b4d009f1f25133ec]) successfully deleted
I20240503 02:09:15.032919 28653 ts_tablet_manager.cc:1888] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.032963 28653 log.cc:1196] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:15.032990 28654 ts_tablet_manager.cc:1875] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.033141 28653 ts_tablet_manager.cc:1909] T 092ad3ae09fe47bbbe473643060c43d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.033409 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.034260 28661 ts_tablet_manager.cc:1888] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.034408 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_679392a2422e43f88ca52faf229d218a [id=0e67e01150414cfda8f556738345b4be] alter complete (version 1)
I20240503 02:09:15.034484 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a93b184492cf412d8e449e8c0527ea61
I20240503 02:09:15.034560 27194 catalog_manager.cc:6554] ReplaceTablet: tablet a93b184492cf412d8e449e8c0527ea61 deleted and replaced by tablet f7c9c3fa8cd1495bb93818b2879ecfd4
I20240503 02:09:15.034417 27203 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_c6bdef5afa174922afb43e03000feebb [id=b70cacd384524dd890671729a21e4fcb] alter complete (version 2)
I20240503 02:09:15.034714 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet a93b184492cf412d8e449e8c0527ea61 with delete_type TABLET_DATA_DELETED (replaced by tablet f7c9c3fa8cd1495bb93818b2879ecfd4) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.034790 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet a93b184492cf412d8e449e8c0527ea61 with delete_type TABLET_DATA_DELETED (replaced by tablet f7c9c3fa8cd1495bb93818b2879ecfd4) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.034905 28653 tablet_replica.cc:330] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.034951 28653 raft_consensus.cc:2229] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.035012 28653 pending_rounds.cc:62] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:15.035038 28653 pending_rounds.cc:68] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420026102956032 op_type: NO_OP noop_request { }
I20240503 02:09:15.035077 28653 raft_consensus.cc:2866] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.035104 28653 raft_consensus.cc:2258] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.035557 28654 ts_tablet_manager.cc:1888] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.035605 28654 log.cc:1196] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:15.035610 28653 ts_tablet_manager.cc:1875] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.034793 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet a93b184492cf412d8e449e8c0527ea61 with delete_type TABLET_DATA_DELETED (replaced by tablet f7c9c3fa8cd1495bb93818b2879ecfd4) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.034858 28706 tablet.cc:1722] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.036681 28653 ts_tablet_manager.cc:1888] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.036723 28653 log.cc:1196] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a93b184492cf412d8e449e8c0527ea61
I20240503 02:09:15.037353 28653 ts_tablet_manager.cc:1909] T a93b184492cf412d8e449e8c0527ea61 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.037468 27203 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_df038b49f4354dc0ba3846a1f0abe3df" } new_table_name: "default.Table_c8acdbb592984df1a1daec4e8588b4ae" modify_external_catalogs: true
I20240503 02:09:15.037464 27194 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dd793faec53545858dfe59dce7a5066c" } new_table_name: "default.Table_8b7ea237bea9428f8653574227fc6436" modify_external_catalogs: true
I20240503 02:09:15.037602 28661 log.cc:1196] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/092ad3ae09fe47bbbe473643060c43d0
I20240503 02:09:15.037642 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b461ce136ff84b40a82e638866255964"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.037722 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b461ce136ff84b40a82e638866255964 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.037649 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a93b184492cf412d8e449e8c0527ea61 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.037784 28661 ts_tablet_manager.cc:1909] T 092ad3ae09fe47bbbe473643060c43d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.038045 28661 tablet_replica.cc:330] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.038105 28661 raft_consensus.cc:2229] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.038136 28661 raft_consensus.cc:2258] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.038230 28661 ts_tablet_manager.cc:1875] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.038246 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.035781 28654 ts_tablet_manager.cc:1909] T 092ad3ae09fe47bbbe473643060c43d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.038911 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet b46bb7a647c64572bcae04d4dcdc1cda from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.038995 28654 tablet_replica.cc:330] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.039060 28654 raft_consensus.cc:2229] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.039098 28654 raft_consensus.cc:2258] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.039098 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 092ad3ae09fe47bbbe473643060c43d0 (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.039188 28654 ts_tablet_manager.cc:1875] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.040460 28704 tablet.cc:1722] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.040604 28706 tablet.cc:1722] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.040786 28661 ts_tablet_manager.cc:1888] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.040830 28661 log.cc:1196] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a93b184492cf412d8e449e8c0527ea61
I20240503 02:09:15.040971 28661 ts_tablet_manager.cc:1909] T a93b184492cf412d8e449e8c0527ea61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.041061 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b46bb7a647c64572bcae04d4dcdc1cda
I20240503 02:09:15.041097 28694 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.041123 27195 catalog_manager.cc:6554] ReplaceTablet: tablet b46bb7a647c64572bcae04d4dcdc1cda deleted and replaced by tablet b8fafdd73d1746d694a46c9f0abb4266
I20240503 02:09:15.041429 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda with delete_type TABLET_DATA_DELETED (replaced by tablet b8fafdd73d1746d694a46c9f0abb4266) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.041684 28654 ts_tablet_manager.cc:1888] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.041729 28654 log.cc:1196] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a93b184492cf412d8e449e8c0527ea61
I20240503 02:09:15.041965 28704 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.042038 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda with delete_type TABLET_DATA_DELETED (replaced by tablet b8fafdd73d1746d694a46c9f0abb4266) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.042222 28661 tablet_replica.cc:330] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.042272 28661 raft_consensus.cc:2229] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.042311 28661 pending_rounds.cc:62] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.042327 28661 pending_rounds.cc:68] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420025357344768 op_type: NO_OP noop_request { }
I20240503 02:09:15.042356 28661 raft_consensus.cc:2866] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.042405 28661 raft_consensus.cc:2258] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.042433 28686 tablet.cc:1722] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.042501 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a93b184492cf412d8e449e8c0527ea61 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.042557 28654 ts_tablet_manager.cc:1909] T a93b184492cf412d8e449e8c0527ea61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.043015 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet b46bb7a647c64572bcae04d4dcdc1cda with delete_type TABLET_DATA_DELETED (replaced by tablet b8fafdd73d1746d694a46c9f0abb4266) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.043020 28661 ts_tablet_manager.cc:1875] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.043081 28710 tablet.cc:1722] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.044026 28653 tablet_replica.cc:330] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.044083 28653 raft_consensus.cc:2229] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.044114 28653 raft_consensus.cc:2258] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.044121 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_c8acdbb592984df1a1daec4e8588b4ae [id=f8d3271eebfc495f8684be5e39d3fb89] alter complete (version 1)
I20240503 02:09:15.044198 28653 ts_tablet_manager.cc:1875] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.044519 28654 tablet_replica.cc:330] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.044584 28654 raft_consensus.cc:2229] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.044620 28654 raft_consensus.cc:2258] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.044750 28654 ts_tablet_manager.cc:1875] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.044821 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a93b184492cf412d8e449e8c0527ea61 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.045315 28653 ts_tablet_manager.cc:1888] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.045382 28653 log.cc:1196] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b46bb7a647c64572bcae04d4dcdc1cda
I20240503 02:09:15.045881 28654 ts_tablet_manager.cc:1888] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.045923 28654 log.cc:1196] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b46bb7a647c64572bcae04d4dcdc1cda
I20240503 02:09:15.046036 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd] alter complete (version 2)
I20240503 02:09:15.046063 28654 ts_tablet_manager.cc:1909] T b46bb7a647c64572bcae04d4dcdc1cda P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.046294 28653 ts_tablet_manager.cc:1909] T b46bb7a647c64572bcae04d4dcdc1cda P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.046474 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_23855b5c24a34d70b99f876e28b2c299" } new_table_name: "default.Table_ee715643ae554caea8c70ede6c01c912" modify_external_catalogs: true
I20240503 02:09:15.046638 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5d89254081104ad3be52d93900293b1b" } modify_external_catalogs: true
I20240503 02:09:15.046656 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b46bb7a647c64572bcae04d4dcdc1cda (table default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.046681 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5d89254081104ad3be52d93900293b1b" } modify_external_catalogs: true
I20240503 02:09:15.046769 28507 raft_consensus.cc:482] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.046808 28507 raft_consensus.cc:504] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.046907 28507 leader_election.cc:288] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.046933 27194 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c6bdef5afa174922afb43e03000feebb" } modify_external_catalogs: true
I20240503 02:09:15.046984 27194 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c6bdef5afa174922afb43e03000feebb" } modify_external_catalogs: true
I20240503 02:09:15.046983 28661 ts_tablet_manager.cc:1888] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.047019 28661 log.cc:1196] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b46bb7a647c64572bcae04d4dcdc1cda
I20240503 02:09:15.047139 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed866637e1fa4b9fbcee3bbd93fd785d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.047184 28661 ts_tablet_manager.cc:1909] T b46bb7a647c64572bcae04d4dcdc1cda P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.047191 27722 raft_consensus.cc:2454] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.047263 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed866637e1fa4b9fbcee3bbd93fd785d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.047322 27451 raft_consensus.cc:2454] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.047382 27523 leader_election.cc:302] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.047453 28507 raft_consensus.cc:2781] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.047479 28507 raft_consensus.cc:482] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.047497 28507 raft_consensus.cc:3037] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.047533 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b46bb7a647c64572bcae04d4dcdc1cda (table default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.047924 28507 raft_consensus.cc:504] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.048013 28507 leader_election.cc:288] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.048161 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed866637e1fa4b9fbcee3bbd93fd785d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.048213 27722 raft_consensus.cc:3037] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.048741 27722 raft_consensus.cc:2454] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.048820 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed866637e1fa4b9fbcee3bbd93fd785d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.048868 27451 raft_consensus.cc:3037] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.048918 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b46bb7a647c64572bcae04d4dcdc1cda (table default.Table_23855b5c24a34d70b99f876e28b2c299 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.048938 27523 leader_election.cc:302] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.049079 28507 raft_consensus.cc:2781] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.049124 28507 raft_consensus.cc:686] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.049158 28507 consensus_queue.cc:237] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.049484 27451 raft_consensus.cc:2454] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.050074 27203 catalog_manager.cc:5484] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.050158 27203 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:15.051187 27199 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c31a2f92fdcf40928ee9e37d49a3ac47 (table default.Table_c8acdbb592984df1a1daec4e8588b4ae [id=f8d3271eebfc495f8684be5e39d3fb89]). Expected version 1 got 0
I20240503 02:09:15.051460 28507 raft_consensus.cc:482] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.051508 28507 raft_consensus.cc:504] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.051604 28507 leader_election.cc:288] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.051764 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b381283603ff4e84a6a399658044a071" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.051823 27722 raft_consensus.cc:2454] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.051889 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b381283603ff4e84a6a399658044a071" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.051936 27451 raft_consensus.cc:2454] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.051970 27523 leader_election.cc:302] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.052039 28507 raft_consensus.cc:2781] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.052067 28507 raft_consensus.cc:482] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.052098 28507 raft_consensus.cc:3037] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.052603 28507 raft_consensus.cc:504] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.052714 28507 leader_election.cc:288] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.052812 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b381283603ff4e84a6a399658044a071" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.052867 27722 raft_consensus.cc:3037] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.053375 27199 master_service.cc:946] ReplaceTablet: received request to replace tablet b381283603ff4e84a6a399658044a071 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.053390 27722 raft_consensus.cc:2454] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.053542 27523 leader_election.cc:302] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.053630 28507 raft_consensus.cc:2781] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.053663 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b381283603ff4e84a6a399658044a071" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.053689 28507 raft_consensus.cc:686] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.053728 27451 raft_consensus.cc:3037] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.053725 28507 consensus_queue.cc:237] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:15.053946 27196 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b8fafdd73d1746d694a46c9f0abb4266 has no consensus state
I20240503 02:09:15.054109 27195 catalog_manager.cc:5766] Not sending DeleteTablet requests; no consensus state for tablet 88d9eeb8fbf34aa88fa17baf0369bce6
I20240503 02:09:15.054158 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1f4bf62a461048fea8dda930b82f464e
I20240503 02:09:15.054246 27451 raft_consensus.cc:2454] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.054417 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f4bf62a461048fea8dda930b82f464e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.054448 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f4bf62a461048fea8dda930b82f464e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.054517 28654 tablet_replica.cc:330] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.054559 28654 raft_consensus.cc:2229] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.054586 28654 raft_consensus.cc:2258] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.054641 28661 tablet_replica.cc:330] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.054693 28661 raft_consensus.cc:2229] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.054706 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f4bf62a461048fea8dda930b82f464e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.054739 28661 raft_consensus.cc:2258] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.054765 28653 tablet_replica.cc:330] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.054800 28654 ts_tablet_manager.cc:1875] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.054813 28653 raft_consensus.cc:2229] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.054845 28653 raft_consensus.cc:2258] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.055100 28653 ts_tablet_manager.cc:1875] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.056207 28653 ts_tablet_manager.cc:1888] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.056217 28654 ts_tablet_manager.cc:1888] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.056244 28661 ts_tablet_manager.cc:1875] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.056257 28653 log.cc:1196] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1f4bf62a461048fea8dda930b82f464e
I20240503 02:09:15.056260 28654 log.cc:1196] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1f4bf62a461048fea8dda930b82f464e
I20240503 02:09:15.056445 28653 ts_tablet_manager.cc:1909] T 1f4bf62a461048fea8dda930b82f464e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.056463 28654 ts_tablet_manager.cc:1909] T 1f4bf62a461048fea8dda930b82f464e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.056602 27199 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b381283603ff4e84a6a399658044a071
I20240503 02:09:15.056703 27199 catalog_manager.cc:6554] ReplaceTablet: tablet b381283603ff4e84a6a399658044a071 deleted and replaced by tablet 694e3a4f847e4f2d9d386cbb5c09842c
I20240503 02:09:15.057000 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet b381283603ff4e84a6a399658044a071 with delete_type TABLET_DATA_DELETED (replaced by tablet 694e3a4f847e4f2d9d386cbb5c09842c) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.057005 27700 tablet_service.cc:1514] Processing DeleteTablet for tablet b381283603ff4e84a6a399658044a071 with delete_type TABLET_DATA_DELETED (replaced by tablet 694e3a4f847e4f2d9d386cbb5c09842c) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.057108 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c6bdef5afa174922afb43e03000feebb [id=b70cacd384524dd890671729a21e4fcb]) successfully deleted
I20240503 02:09:15.057190 28653 tablet_replica.cc:330] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.057240 28653 raft_consensus.cc:2229] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.057246 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet b381283603ff4e84a6a399658044a071 with delete_type TABLET_DATA_DELETED (replaced by tablet 694e3a4f847e4f2d9d386cbb5c09842c) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.057269 28653 raft_consensus.cc:2258] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.057315 28654 tablet_replica.cc:330] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.057387 28654 raft_consensus.cc:2229] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.057426 28654 pending_rounds.cc:62] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:15.057441 28654 pending_rounds.cc:68] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027100360704 op_type: NO_OP noop_request { }
I20240503 02:09:15.057469 28654 raft_consensus.cc:2866] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.057483 28654 raft_consensus.cc:2258] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.057562 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c6bdef5afa174922afb43e03000feebb [id=b70cacd384524dd890671729a21e4fcb]) successfully deleted
I20240503 02:09:15.057662 28653 ts_tablet_manager.cc:1875] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.057704 28654 ts_tablet_manager.cc:1875] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.058463 28661 ts_tablet_manager.cc:1888] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.058568 28661 log.cc:1196] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1f4bf62a461048fea8dda930b82f464e
I20240503 02:09:15.058743 28653 ts_tablet_manager.cc:1888] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.058784 28653 log.cc:1196] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b381283603ff4e84a6a399658044a071
I20240503 02:09:15.058861 28654 ts_tablet_manager.cc:1888] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.058892 28661 ts_tablet_manager.cc:1909] T 1f4bf62a461048fea8dda930b82f464e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.058904 28654 log.cc:1196] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b381283603ff4e84a6a399658044a071
I20240503 02:09:15.058943 28653 ts_tablet_manager.cc:1909] T b381283603ff4e84a6a399658044a071 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.059120 28661 tablet_replica.cc:330] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.059140 28654 ts_tablet_manager.cc:1909] T b381283603ff4e84a6a399658044a071 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.059165 28661 raft_consensus.cc:2229] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.059193 28661 raft_consensus.cc:2258] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.059219 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1f4bf62a461048fea8dda930b82f464e (table default.Table_c6bdef5afa174922afb43e03000feebb [id=b70cacd384524dd890671729a21e4fcb]) successfully deleted
I20240503 02:09:15.059556 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b381283603ff4e84a6a399658044a071 (table default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:15.059808 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b381283603ff4e84a6a399658044a071 (table default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:15.060004 28661 ts_tablet_manager.cc:1875] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.060828 27582 raft_consensus.cc:1261] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.060992 27451 raft_consensus.cc:1261] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.061003 28740 consensus_queue.cc:1035] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.061556 28661 ts_tablet_manager.cc:1888] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.061605 28740 consensus_queue.cc:1035] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.061599 28661 log.cc:1196] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b381283603ff4e84a6a399658044a071
I20240503 02:09:15.062096 28661 ts_tablet_manager.cc:1909] T b381283603ff4e84a6a399658044a071 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.062480 28706 tablet.cc:1722] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.062638 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b381283603ff4e84a6a399658044a071 (table default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:15.062628 28686 tablet.cc:1722] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.063560 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.063906 27199 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:15.064622 27563 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (DEFAULT_TABLE table=default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064707 27564 tablet_service.cc:1467] Processing CreateTablet for tablet 43e8dd055a1d4673811f6e775340f008 (DEFAULT_TABLE table=default.Table_eec82c477df84e9dbd2b9874505d9e48 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064739 27562 tablet_service.cc:1467] Processing CreateTablet for tablet e3c9273bebae49509f713f02bc050cf5 (DEFAULT_TABLE table=default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064808 27560 tablet_service.cc:1467] Processing CreateTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 (DEFAULT_TABLE table=default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064862 27565 tablet_service.cc:1467] Processing CreateTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf (DEFAULT_TABLE table=default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064913 27567 tablet_service.cc:1467] Processing CreateTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064960 27552 tablet_service.cc:1467] Processing CreateTablet for tablet 33d313988a8c418ea82af994d23e7e15 (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.064981 27701 tablet_service.cc:1467] Processing CreateTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf (DEFAULT_TABLE table=default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065002 27557 tablet_service.cc:1467] Processing CreateTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 (DEFAULT_TABLE table=default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065048 27556 tablet_service.cc:1467] Processing CreateTablet for tablet b280b918700342319e9484cdb1c4b949 (DEFAULT_TABLE table=default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0 [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065066 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b8f82c500549d29c0d1a41d8e111cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.065089 27555 tablet_service.cc:1467] Processing CreateTablet for tablet 78d80210ca3842fba4243bafc7498b2a (DEFAULT_TABLE table=default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065135 27553 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc07eb64969465fb2b152337dc82851 (DEFAULT_TABLE table=default.Table_37701508cc6c4db8b07876a048312845 [id=dcaa66ed9577423ebe462a9302aa4a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065182 27554 tablet_service.cc:1467] Processing CreateTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065186 27695 tablet_service.cc:1467] Processing CreateTablet for tablet 33d313988a8c418ea82af994d23e7e15 (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065229 27558 tablet_service.cc:1467] Processing CreateTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065263 27695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d313988a8c418ea82af994d23e7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.065368 27700 tablet_service.cc:1467] Processing CreateTablet for tablet e3c9273bebae49509f713f02bc050cf5 (DEFAULT_TABLE table=default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065393 27698 tablet_service.cc:1467] Processing CreateTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.065433 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c9273bebae49509f713f02bc050cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.065477 27698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e3a4f847e4f2d9d386cbb5c09842c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.066332 27696 tablet_service.cc:1467] Processing CreateTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 (DEFAULT_TABLE table=default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.066356 27703 tablet_service.cc:1467] Processing CreateTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 (DEFAULT_TABLE table=default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.066406 27696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8fafdd73d1746d694a46c9f0abb4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.066434 27699 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (DEFAULT_TABLE table=default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.066504 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce6b30fc5c742579f061c4bc50ecdf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.067399 27702 tablet_service.cc:1467] Processing CreateTablet for tablet b280b918700342319e9484cdb1c4b949 (DEFAULT_TABLE table=default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0 [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.067474 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b280b918700342319e9484cdb1c4b949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.067481 28664 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.067659 27722 raft_consensus.cc:1261] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.067756 28664 tablet_bootstrap.cc:654] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.067836 27848 consensus_queue.cc:1035] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.068280 27429 tablet_service.cc:1467] Processing CreateTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c (DEFAULT_TABLE table=default.Table_acc60625100f46ca9a4b6b8ee71f3a4b [id=1d806aada5814e4d86568e5488045240]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068324 27431 tablet_service.cc:1467] Processing CreateTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 (DEFAULT_TABLE table=default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068365 27422 tablet_service.cc:1467] Processing CreateTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 (DEFAULT_TABLE table=default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068400 27426 tablet_service.cc:1467] Processing CreateTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068282 27424 tablet_service.cc:1467] Processing CreateTablet for tablet e3c9273bebae49509f713f02bc050cf5 (DEFAULT_TABLE table=default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068305 27432 tablet_service.cc:1467] Processing CreateTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf (DEFAULT_TABLE table=default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068470 27423 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (DEFAULT_TABLE table=default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068534 27425 tablet_service.cc:1467] Processing CreateTablet for tablet b280b918700342319e9484cdb1c4b949 (DEFAULT_TABLE table=default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0 [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068529 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 78d80210ca3842fba4243bafc7498b2a (DEFAULT_TABLE table=default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068596 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d80210ca3842fba4243bafc7498b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.068603 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 43e8dd055a1d4673811f6e775340f008 (DEFAULT_TABLE table=default.Table_eec82c477df84e9dbd2b9874505d9e48 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068403 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e3a4f847e4f2d9d386cbb5c09842c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.068708 27433 tablet_service.cc:1467] Processing CreateTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068382 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 78d80210ca3842fba4243bafc7498b2a (DEFAULT_TABLE table=default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068796 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd5396b5f8041959f7e2ed462fbf825. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.069499 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd5396b5f8041959f7e2ed462fbf825. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.069614 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 43e8dd055a1d4673811f6e775340f008 (DEFAULT_TABLE table=default.Table_eec82c477df84e9dbd2b9874505d9e48 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.068652 27420 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc07eb64969465fb2b152337dc82851 (DEFAULT_TABLE table=default.Table_37701508cc6c4db8b07876a048312845 [id=dcaa66ed9577423ebe462a9302aa4a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.069685 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e8dd055a1d4673811f6e775340f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.070721 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 9dc07eb64969465fb2b152337dc82851 (DEFAULT_TABLE table=default.Table_37701508cc6c4db8b07876a048312845 [id=dcaa66ed9577423ebe462a9302aa4a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.070799 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc07eb64969465fb2b152337dc82851. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.071729 27691 tablet_service.cc:1467] Processing CreateTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.071761 28655 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.071803 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac78bd3c0c254bc9b46a11df891d2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.072095 28655 tablet_bootstrap.cc:654] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.072507 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c9273bebae49509f713f02bc050cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.072782 27690 tablet_service.cc:1467] Processing CreateTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.072853 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7c9c3fa8cd1495bb93818b2879ecfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.073552 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b8f82c500549d29c0d1a41d8e111cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.073760 27848 raft_consensus.cc:482] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.073807 27848 raft_consensus.cc:504] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.073866 27421 tablet_service.cc:1467] Processing CreateTablet for tablet 33d313988a8c418ea82af994d23e7e15 (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.073894 27848 leader_election.cc:288] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.073944 27421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d313988a8c418ea82af994d23e7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.074021 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cbd420b867b4144985103d498b1ea73" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.074079 27449 raft_consensus.cc:2454] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.074195 27524 leader_election.cc:302] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.074255 27848 raft_consensus.cc:2781] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.074282 27848 raft_consensus.cc:482] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.074316 27848 raft_consensus.cc:3037] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.074580 27423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce6b30fc5c742579f061c4bc50ecdf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.074868 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cbd420b867b4144985103d498b1ea73" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.074878 27848 raft_consensus.cc:504] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.074921 27721 raft_consensus.cc:2454] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.064720 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce6b30fc5c742579f061c4bc50ecdf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.074980 27848 leader_election.cc:288] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.075026 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8fafdd73d1746d694a46c9f0abb4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.075086 27552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33d313988a8c418ea82af994d23e7e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.075359 27564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e8dd055a1d4673811f6e775340f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.075659 27431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8fafdd73d1746d694a46c9f0abb4266. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.076169 27556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b280b918700342319e9484cdb1c4b949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.076323 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e3a4f847e4f2d9d386cbb5c09842c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.076447 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cbd420b867b4144985103d498b1ea73" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.076500 27721 raft_consensus.cc:3037] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.076622 27420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc07eb64969465fb2b152337dc82851. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.077030 27721 raft_consensus.cc:2454] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.077425 27523 leader_election.cc:302] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.077508 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cbd420b867b4144985103d498b1ea73" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.077543 27555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d80210ca3842fba4243bafc7498b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.077556 27449 raft_consensus.cc:3037] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.077735 28656 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.078090 27449 raft_consensus.cc:2454] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.078147 28656 tablet_bootstrap.cc:654] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.078269 27451 raft_consensus.cc:1261] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.078419 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcd5396b5f8041959f7e2ed462fbf825. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.078475 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dc07eb64969465fb2b152337dc82851. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.078550 28507 raft_consensus.cc:2781] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.078586 28507 raft_consensus.cc:686] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.078619 28507 consensus_queue.cc:237] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.078660 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c9273bebae49509f713f02bc050cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.078765 28507 consensus_queue.cc:1035] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.079252 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac78bd3c0c254bc9b46a11df891d2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.079633 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b8f82c500549d29c0d1a41d8e111cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.080350 28656 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.080392 28656 ts_tablet_manager.cc:1367] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.080510 28656 raft_consensus.cc:348] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.080554 28656 raft_consensus.cc:374] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.080571 28656 raft_consensus.cc:729] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.080597 28656 consensus_queue.cc:260] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.080686 28656 ts_tablet_manager.cc:1398] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.080731 28656 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.080755 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7c9c3fa8cd1495bb93818b2879ecfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.080978 27195 catalog_manager.cc:5484] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.081065 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:15.081291 27199 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:15.081713 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:15.081930 28655 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.081966 28655 ts_tablet_manager.cc:1367] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:15.081992 28656 tablet_bootstrap.cc:654] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.082075 28655 raft_consensus.cc:348] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.082126 28655 raft_consensus.cc:374] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.082144 28655 raft_consensus.cc:729] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.082180 28655 consensus_queue.cc:260] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.082258 28655 ts_tablet_manager.cc:1398] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.082302 28655 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.082346 28698 tablet.cc:1722] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.082607 28655 tablet_bootstrap.cc:654] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.083045 28656 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.083079 28656 ts_tablet_manager.cc:1367] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.083192 28656 raft_consensus.cc:348] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.083237 28656 raft_consensus.cc:374] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.083253 28656 raft_consensus.cc:729] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.083297 28656 consensus_queue.cc:260] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.083374 28656 ts_tablet_manager.cc:1398] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.083416 28656 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.083724 28656 tablet_bootstrap.cc:654] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.083858 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7c9c3fa8cd1495bb93818b2879ecfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.084437 28686 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.084832 28706 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.084905 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78d80210ca3842fba4243bafc7498b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.085639 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet ed866637e1fa4b9fbcee3bbd93fd785d (table default.Table_63ccfec42f064aacaf0c950ec3442742 [id=c46a0a1c65dd47e38382f2b2d19bff05]). Expected version 2 got 0
I20240503 02:09:15.085970 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac78bd3c0c254bc9b46a11df891d2c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.086298 28912 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.086990 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b280b918700342319e9484cdb1c4b949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.087790 28656 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.087827 28656 ts_tablet_manager.cc:1367] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.087949 28656 raft_consensus.cc:348] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.087997 28656 raft_consensus.cc:374] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.088014 28656 raft_consensus.cc:729] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.088047 28656 consensus_queue.cc:260] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.088105 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43e8dd055a1d4673811f6e775340f008. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.088124 28656 ts_tablet_manager.cc:1398] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.088171 28656 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.088595 28656 tablet_bootstrap.cc:654] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.089051 28656 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.089085 28656 ts_tablet_manager.cc:1367] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.089190 28656 raft_consensus.cc:348] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.089239 28656 raft_consensus.cc:374] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.089254 28656 raft_consensus.cc:729] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.089299 28656 consensus_queue.cc:260] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.089439 28656 ts_tablet_manager.cc:1398] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.089491 28656 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.089794 28656 tablet_bootstrap.cc:654] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.090339 28664 tablet_bootstrap.cc:492] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.090360 28656 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.090384 28664 ts_tablet_manager.cc:1367] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240503 02:09:15.090399 28656 ts_tablet_manager.cc:1367] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.090497 28664 raft_consensus.cc:348] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.090507 28656 raft_consensus.cc:348] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.090544 28664 raft_consensus.cc:374] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.090552 28656 raft_consensus.cc:374] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.090562 28664 raft_consensus.cc:729] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.090570 28656 raft_consensus.cc:729] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.090595 28656 consensus_queue.cc:260] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.090597 28664 consensus_queue.cc:260] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.090665 28656 ts_tablet_manager.cc:1398] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.090682 28664 ts_tablet_manager.cc:1398] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.090749 28664 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.090802 28656 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.091140 28664 tablet_bootstrap.cc:654] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.091147 28656 tablet_bootstrap.cc:654] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.092026 28655 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.092077 28655 ts_tablet_manager.cc:1367] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:15.092195 28655 raft_consensus.cc:348] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.092248 28655 raft_consensus.cc:374] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.092267 28655 raft_consensus.cc:729] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.092300 28655 consensus_queue.cc:260] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.092387 28655 ts_tablet_manager.cc:1398] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.092458 28655 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.092607 28656 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.092643 28656 ts_tablet_manager.cc:1367] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.092761 28655 tablet_bootstrap.cc:654] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.092757 28656 raft_consensus.cc:348] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.092810 28656 raft_consensus.cc:374] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.092828 28656 raft_consensus.cc:729] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.092866 28656 consensus_queue.cc:260] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.092952 28656 ts_tablet_manager.cc:1398] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.092999 28656 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.093233 28655 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.093286 28655 ts_tablet_manager.cc:1367] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.093286 28656 tablet_bootstrap.cc:654] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.093441 28655 raft_consensus.cc:348] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.093497 28655 raft_consensus.cc:374] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.093515 28655 raft_consensus.cc:729] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.093547 28655 consensus_queue.cc:260] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.093636 28655 ts_tablet_manager.cc:1398] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.093699 28655 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.094007 28655 tablet_bootstrap.cc:654] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.094120 28664 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.094157 28664 ts_tablet_manager.cc:1367] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:15.094287 28664 raft_consensus.cc:348] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.094348 28664 raft_consensus.cc:374] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.094370 28664 raft_consensus.cc:729] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.094403 28664 consensus_queue.cc:260] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.094491 28664 ts_tablet_manager.cc:1398] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.094552 28664 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.094571 28656 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.094609 28656 ts_tablet_manager.cc:1367] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.094810 28656 raft_consensus.cc:348] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.094861 28664 tablet_bootstrap.cc:654] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.094868 28656 raft_consensus.cc:374] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.094887 28656 raft_consensus.cc:729] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.094921 28656 consensus_queue.cc:260] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.095000 28656 ts_tablet_manager.cc:1398] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.095048 28656 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.095185 28655 tablet_bootstrap.cc:492] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.095211 27199 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:15.095220 28655 ts_tablet_manager.cc:1367] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
W20240503 02:09:15.095275 27199 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 78d80210ca3842fba4243bafc7498b2a consensus state has no leader
I20240503 02:09:15.095361 28656 tablet_bootstrap.cc:654] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.095352 28655 raft_consensus.cc:348] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.095400 28655 raft_consensus.cc:374] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.095419 28655 raft_consensus.cc:729] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.095453 28655 consensus_queue.cc:260] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.095523 28655 ts_tablet_manager.cc:1398] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.095564 28655 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.095870 28655 tablet_bootstrap.cc:654] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.096264 28656 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.096307 28656 ts_tablet_manager.cc:1367] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.096432 28656 raft_consensus.cc:348] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.096480 28656 raft_consensus.cc:374] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.096498 28656 raft_consensus.cc:729] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.096529 28656 consensus_queue.cc:260] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.096637 28656 ts_tablet_manager.cc:1398] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.096680 28656 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.096973 28656 tablet_bootstrap.cc:654] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.097365 28664 tablet_bootstrap.cc:492] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.097415 28664 ts_tablet_manager.cc:1367] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:15.097561 28664 raft_consensus.cc:348] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.097626 28664 raft_consensus.cc:374] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.097643 28664 raft_consensus.cc:729] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.097676 28664 consensus_queue.cc:260] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.097781 28664 ts_tablet_manager.cc:1398] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.097860 28664 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.098186 28664 tablet_bootstrap.cc:654] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.098515 28656 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.098565 28656 ts_tablet_manager.cc:1367] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.098671 28656 raft_consensus.cc:348] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.098722 28656 raft_consensus.cc:374] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.098739 28656 raft_consensus.cc:729] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.098769 28656 consensus_queue.cc:260] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.098862 28656 ts_tablet_manager.cc:1398] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.098912 28656 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.098918 28664 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.098955 28664 ts_tablet_manager.cc:1367] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.099025 28655 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.099061 28655 ts_tablet_manager.cc:1367] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.099067 28664 raft_consensus.cc:348] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.099117 28664 raft_consensus.cc:374] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.099133 28664 raft_consensus.cc:729] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.099174 28664 consensus_queue.cc:260] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.099201 28656 tablet_bootstrap.cc:654] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.099247 28664 ts_tablet_manager.cc:1398] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.099236 28655 raft_consensus.cc:348] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.099298 28664 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.099298 28655 raft_consensus.cc:374] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.099316 28655 raft_consensus.cc:729] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.099350 28655 consensus_queue.cc:260] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.099427 28655 ts_tablet_manager.cc:1398] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.099483 28655 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.099611 28664 tablet_bootstrap.cc:654] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.099783 28655 tablet_bootstrap.cc:654] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.100083 28656 tablet_bootstrap.cc:492] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.100123 28656 ts_tablet_manager.cc:1367] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.100188 28774 raft_consensus.cc:482] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.100232 28774 raft_consensus.cc:504] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.100247 28656 raft_consensus.cc:348] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.100296 28656 raft_consensus.cc:374] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.100313 28656 raft_consensus.cc:729] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.100345 28656 consensus_queue.cc:260] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.100430 28656 ts_tablet_manager.cc:1398] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.100473 28656 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.100517 28774 leader_election.cc:288] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.100739 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c583b44cecad4e93a50fddae3a0035b7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.100802 27582 raft_consensus.cc:2454] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.100929 28656 tablet_bootstrap.cc:654] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.100942 27656 leader_election.cc:302] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.101089 28774 raft_consensus.cc:2781] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.101121 28774 raft_consensus.cc:482] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.101097 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c583b44cecad4e93a50fddae3a0035b7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.101138 28774 raft_consensus.cc:3037] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.101146 27451 raft_consensus.cc:2454] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.101783 28774 raft_consensus.cc:504] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.101809 28656 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.101850 28656 ts_tablet_manager.cc:1367] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.101882 28774 leader_election.cc:288] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.101976 28656 raft_consensus.cc:348] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.102023 28656 raft_consensus.cc:374] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.102041 28656 raft_consensus.cc:729] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.102085 28656 consensus_queue.cc:260] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.102154 28656 ts_tablet_manager.cc:1398] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.102192 28656 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.102250 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c583b44cecad4e93a50fddae3a0035b7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.102295 27582 raft_consensus.cc:3037] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.102475 28656 tablet_bootstrap.cc:654] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.102502 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c583b44cecad4e93a50fddae3a0035b7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.102557 27451 raft_consensus.cc:3037] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.102864 27582 raft_consensus.cc:2454] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.102994 27656 leader_election.cc:302] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.103065 28774 raft_consensus.cc:2781] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.103109 28774 raft_consensus.cc:686] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.103125 27451 raft_consensus.cc:2454] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.103142 28774 consensus_queue.cc:237] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.103330 28656 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.103367 28656 ts_tablet_manager.cc:1367] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.103477 28656 raft_consensus.cc:348] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.103535 28656 raft_consensus.cc:374] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.103552 28656 raft_consensus.cc:729] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.103590 28656 consensus_queue.cc:260] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.103670 28656 ts_tablet_manager.cc:1398] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.103716 28656 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.104023 28656 tablet_bootstrap.cc:654] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.104103 28655 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.104152 28655 ts_tablet_manager.cc:1367] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:15.104274 28655 raft_consensus.cc:348] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.104337 28655 raft_consensus.cc:374] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.104357 28655 raft_consensus.cc:729] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.104396 28655 consensus_queue.cc:260] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.104470 28655 ts_tablet_manager.cc:1398] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.104513 28655 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.104817 28655 tablet_bootstrap.cc:654] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.105212 28656 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.105176 27194 catalog_manager.cc:5484] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.105249 28656 ts_tablet_manager.cc:1367] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.105376 28656 raft_consensus.cc:348] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.105439 28656 raft_consensus.cc:374] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.105458 28656 raft_consensus.cc:729] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.105491 28656 consensus_queue.cc:260] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.105583 28656 ts_tablet_manager.cc:1398] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.105580 28664 tablet_bootstrap.cc:492] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.105634 28664 ts_tablet_manager.cc:1367] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:15.105752 28664 raft_consensus.cc:348] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.105819 28664 raft_consensus.cc:374] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.105842 28664 raft_consensus.cc:729] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.105872 28664 consensus_queue.cc:260] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.105942 28664 ts_tablet_manager.cc:1398] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.105960 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]). Expected version 1 got 0
I20240503 02:09:15.105993 28664 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.106302 28664 tablet_bootstrap.cc:654] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.106590 28655 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.106673 28655 ts_tablet_manager.cc:1367] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.106801 28655 raft_consensus.cc:348] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.106853 28655 raft_consensus.cc:374] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.106868 28655 raft_consensus.cc:729] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.107282 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]). Expected version 1 got 0
I20240503 02:09:15.106904 28655 consensus_queue.cc:260] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.107646 28655 ts_tablet_manager.cc:1398] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.107743 28655 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.108039 28655 tablet_bootstrap.cc:654] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.109149 28664 tablet_bootstrap.cc:492] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.109189 28664 ts_tablet_manager.cc:1367] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.109304 28664 raft_consensus.cc:348] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.109377 28664 raft_consensus.cc:374] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.109400 28664 raft_consensus.cc:729] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.109437 28664 consensus_queue.cc:260] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.109532 28664 ts_tablet_manager.cc:1398] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.109607 28664 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.109611 28655 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.109649 28655 ts_tablet_manager.cc:1367] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.109776 28655 raft_consensus.cc:348] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.109829 28655 raft_consensus.cc:374] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.109848 28655 raft_consensus.cc:729] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.109892 28655 consensus_queue.cc:260] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.109910 28664 tablet_bootstrap.cc:654] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.109983 28655 ts_tablet_manager.cc:1398] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.109995 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]). Expected version 1 got 0
I20240503 02:09:15.110049 28655 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.110426 28655 tablet_bootstrap.cc:654] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.110821 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]). Expected version 1 got 0
I20240503 02:09:15.111491 28664 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.111531 28664 ts_tablet_manager.cc:1367] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.111654 28664 raft_consensus.cc:348] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.111697 28664 raft_consensus.cc:374] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.111714 28664 raft_consensus.cc:729] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.111752 28664 consensus_queue.cc:260] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.111832 28664 ts_tablet_manager.cc:1398] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.111917 28664 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.112054 28655 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.112090 28655 ts_tablet_manager.cc:1367] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.112205 28655 raft_consensus.cc:348] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.112254 28664 tablet_bootstrap.cc:654] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.112267 28655 raft_consensus.cc:374] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.112293 28655 raft_consensus.cc:729] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.112329 28655 consensus_queue.cc:260] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.112411 28655 ts_tablet_manager.cc:1398] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.112453 28655 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.112727 28655 tablet_bootstrap.cc:654] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.112746 28664 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.112787 28664 ts_tablet_manager.cc:1367] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.112987 28664 raft_consensus.cc:348] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.113047 28664 raft_consensus.cc:374] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.113065 28664 raft_consensus.cc:729] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.113102 28664 consensus_queue.cc:260] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.113193 28664 ts_tablet_manager.cc:1398] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.113253 28664 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.113587 28664 tablet_bootstrap.cc:654] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.113592 28655 tablet_bootstrap.cc:492] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.113633 28655 ts_tablet_manager.cc:1367] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.113757 28655 raft_consensus.cc:348] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.113814 28655 raft_consensus.cc:374] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.113831 28655 raft_consensus.cc:729] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.113894 28655 consensus_queue.cc:260] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.114023 28655 ts_tablet_manager.cc:1398] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.114099 28655 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.114146 28664 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.114184 28664 ts_tablet_manager.cc:1367] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.114327 28664 raft_consensus.cc:348] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.114351 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:15.114377 28664 raft_consensus.cc:374] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.114394 28664 raft_consensus.cc:729] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.114427 28664 consensus_queue.cc:260] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.114502 28664 ts_tablet_manager.cc:1398] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.114543 28664 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.114830 28655 tablet_bootstrap.cc:654] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.114847 28664 tablet_bootstrap.cc:654] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.115367 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:15.115919 28664 tablet_bootstrap.cc:492] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.115964 28664 ts_tablet_manager.cc:1367] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.116081 28664 raft_consensus.cc:348] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.116135 28664 raft_consensus.cc:374] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.116153 28664 raft_consensus.cc:729] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.116189 28664 consensus_queue.cc:260] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.116284 28664 ts_tablet_manager.cc:1398] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.116345 28664 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.116662 28664 tablet_bootstrap.cc:654] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.117201 28664 tablet_bootstrap.cc:492] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.117247 28664 ts_tablet_manager.cc:1367] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.117403 28664 raft_consensus.cc:348] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.117460 28664 raft_consensus.cc:374] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.117477 28664 raft_consensus.cc:729] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.117523 28664 consensus_queue.cc:260] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.117648 28664 ts_tablet_manager.cc:1398] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.117727 28664 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.118149 28664 tablet_bootstrap.cc:654] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.118809 28664 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.118849 28664 ts_tablet_manager.cc:1367] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.118964 28664 raft_consensus.cc:348] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.119233 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]). Expected version 2 got 0
I20240503 02:09:15.119271 28664 raft_consensus.cc:374] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.119297 28664 raft_consensus.cc:729] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.119339 28655 tablet_bootstrap.cc:492] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.119351 28664 consensus_queue.cc:260] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.119377 28655 ts_tablet_manager.cc:1367] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:15.119479 28664 ts_tablet_manager.cc:1398] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.119504 28655 raft_consensus.cc:348] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.119539 28664 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.119553 28655 raft_consensus.cc:374] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.119573 28655 raft_consensus.cc:729] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.119601 28655 consensus_queue.cc:260] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.119674 28655 ts_tablet_manager.cc:1398] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.119741 28655 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.120046 28664 tablet_bootstrap.cc:654] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.120069 28655 tablet_bootstrap.cc:654] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.120651 28655 tablet_bootstrap.cc:492] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.120688 28655 ts_tablet_manager.cc:1367] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.120806 28655 raft_consensus.cc:348] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.120851 28655 raft_consensus.cc:374] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.120883 28655 raft_consensus.cc:729] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.120919 28655 consensus_queue.cc:260] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.121028 28655 ts_tablet_manager.cc:1398] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.121075 28655 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.121228 28664 tablet_bootstrap.cc:492] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.121315 28664 ts_tablet_manager.cc:1367] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.121404 28655 tablet_bootstrap.cc:654] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.121454 28664 raft_consensus.cc:348] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.121500 28664 raft_consensus.cc:374] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.121515 28664 raft_consensus.cc:729] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.121560 28664 consensus_queue.cc:260] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.121914 28655 tablet_bootstrap.cc:492] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.121958 28655 ts_tablet_manager.cc:1367] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.122081 28655 raft_consensus.cc:348] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.122129 28655 raft_consensus.cc:374] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.122145 28655 raft_consensus.cc:729] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.122179 28655 consensus_queue.cc:260] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.122267 28655 ts_tablet_manager.cc:1398] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.122462 28664 ts_tablet_manager.cc:1398] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.127692 28774 raft_consensus.cc:482] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.127741 28774 raft_consensus.cc:504] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.127856 28774 leader_election.cc:288] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.127979 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b8f82c500549d29c0d1a41d8e111cf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.127986 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b8f82c500549d29c0d1a41d8e111cf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.128036 27451 raft_consensus.cc:2454] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.128038 27582 raft_consensus.cc:2454] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.128154 27656 leader_election.cc:302] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.128242 28774 raft_consensus.cc:2781] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.128271 28774 raft_consensus.cc:482] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.128288 28774 raft_consensus.cc:3037] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.128804 28774 raft_consensus.cc:504] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.128911 28774 leader_election.cc:288] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.129040 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b8f82c500549d29c0d1a41d8e111cf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.129112 27451 raft_consensus.cc:3037] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.129101 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b8f82c500549d29c0d1a41d8e111cf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.129153 27582 raft_consensus.cc:3037] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.129652 27451 raft_consensus.cc:2454] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.129750 27582 raft_consensus.cc:2454] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.129798 27659 leader_election.cc:302] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.129869 28774 raft_consensus.cc:2781] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.129913 28774 raft_consensus.cc:686] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.129953 28774 consensus_queue.cc:237] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.130517 27195 catalog_manager.cc:5484] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.135042 28774 consensus_queue.cc:1035] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.136731 28774 consensus_queue.cc:1035] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.142247 28774 consensus_queue.cc:1035] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.143855 28774 consensus_queue.cc:1035] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.155200 27848 consensus_queue.cc:1035] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.156526 28507 consensus_queue.cc:1035] T c351b66df551435ea01295f1f8dabd25 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.157531 27721 raft_consensus.cc:1261] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.157688 27451 raft_consensus.cc:1261] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.157697 28507 consensus_queue.cc:1035] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.157904 28507 consensus_queue.cc:1035] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.159009 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:15.159065 28698 tablet.cc:1722] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.159229 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:15.159368 28706 tablet.cc:1722] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.159732 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]). Expected version 1 got 0
I20240503 02:09:15.160457 28686 tablet.cc:1722] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.161890 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746] alter complete (version 1)
I20240503 02:09:15.167378 28766 raft_consensus.cc:482] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.167428 28766 raft_consensus.cc:504] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.167536 28766 leader_election.cc:288] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.167680 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d313988a8c418ea82af994d23e7e15" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.167750 27721 raft_consensus.cc:2454] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.167815 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d313988a8c418ea82af994d23e7e15" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.167873 27582 raft_consensus.cc:2454] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.167882 27388 leader_election.cc:302] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.167963 28766 raft_consensus.cc:2781] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.167994 28766 raft_consensus.cc:482] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.168012 28766 raft_consensus.cc:3037] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.168402 28774 raft_consensus.cc:482] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.168449 28774 raft_consensus.cc:504] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.168551 28774 leader_election.cc:288] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.168526 28766 raft_consensus.cc:504] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.168622 28766 leader_election.cc:288] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.168651 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac78bd3c0c254bc9b46a11df891d2c5b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.168704 27451 raft_consensus.cc:2454] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.168741 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac78bd3c0c254bc9b46a11df891d2c5b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.168767 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d313988a8c418ea82af994d23e7e15" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.168805 27587 raft_consensus.cc:2454] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.168864 27659 leader_election.cc:302] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.168839 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33d313988a8c418ea82af994d23e7e15" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.168813 27582 raft_consensus.cc:3037] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.168918 27721 raft_consensus.cc:3037] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.168974 28774 raft_consensus.cc:2781] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.168998 28774 raft_consensus.cc:482] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.169014 28774 raft_consensus.cc:3037] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.169348 27582 raft_consensus.cc:2454] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.169492 27386 leader_election.cc:302] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.169520 27721 raft_consensus.cc:2454] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.169600 28766 raft_consensus.cc:2781] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.169584 28774 raft_consensus.cc:504] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.169649 28766 raft_consensus.cc:686] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.169688 28774 leader_election.cc:288] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.169692 28766 consensus_queue.cc:237] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.169852 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac78bd3c0c254bc9b46a11df891d2c5b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.169914 27582 raft_consensus.cc:3037] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.169957 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac78bd3c0c254bc9b46a11df891d2c5b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.170009 27451 raft_consensus.cc:3037] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.170375 27582 raft_consensus.cc:2454] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.170396 27451 raft_consensus.cc:2454] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.170517 27656 leader_election.cc:302] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.170593 28774 raft_consensus.cc:2781] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.170642 28774 raft_consensus.cc:686] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.170678 28774 consensus_queue.cc:237] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.170791 27199 catalog_manager.cc:5484] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.171838 27194 catalog_manager.cc:5484] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.172719 28774 raft_consensus.cc:482] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.172767 28774 raft_consensus.cc:504] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.172941 28774 leader_election.cc:288] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.172952 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b280b918700342319e9484cdb1c4b949" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.173012 27451 raft_consensus.cc:2454] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.173049 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b280b918700342319e9484cdb1c4b949" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.173117 27582 raft_consensus.cc:2454] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.173112 27659 leader_election.cc:302] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.173177 28774 raft_consensus.cc:2781] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.173209 28774 raft_consensus.cc:482] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.173225 28774 raft_consensus.cc:3037] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.173664 28774 raft_consensus.cc:504] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.173754 28774 leader_election.cc:288] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.173862 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b280b918700342319e9484cdb1c4b949" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.173923 27451 raft_consensus.cc:3037] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.173913 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b280b918700342319e9484cdb1c4b949" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.173974 27582 raft_consensus.cc:3037] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.174312 27451 raft_consensus.cc:2454] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.174435 27582 raft_consensus.cc:2454] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.174448 27659 leader_election.cc:302] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.174566 28774 raft_consensus.cc:2781] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.174608 28774 raft_consensus.cc:686] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.174641 28774 consensus_queue.cc:237] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.175283 27194 catalog_manager.cc:5484] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.180909 28507 raft_consensus.cc:482] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.180958 28507 raft_consensus.cc:504] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.181035 28507 leader_election.cc:288] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.181167 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd5396b5f8041959f7e2ed462fbf825" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.181178 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd5396b5f8041959f7e2ed462fbf825" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.181238 27451 raft_consensus.cc:2454] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.181238 27721 raft_consensus.cc:2454] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.181371 27524 leader_election.cc:302] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.181465 28507 raft_consensus.cc:2781] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.181515 28507 raft_consensus.cc:482] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.181533 28507 raft_consensus.cc:3037] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.181941 28507 raft_consensus.cc:504] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.182049 28507 leader_election.cc:288] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.182193 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd5396b5f8041959f7e2ed462fbf825" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.182250 27451 raft_consensus.cc:3037] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.182245 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcd5396b5f8041959f7e2ed462fbf825" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.182289 27721 raft_consensus.cc:3037] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.182793 27721 raft_consensus.cc:2454] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.182790 27451 raft_consensus.cc:2454] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.182976 27524 leader_election.cc:302] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.183074 28507 raft_consensus.cc:2781] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.183120 28507 raft_consensus.cc:686] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.183162 28507 consensus_queue.cc:237] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.183722 27194 catalog_manager.cc:5484] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.190167 28507 raft_consensus.cc:482] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.190217 28507 raft_consensus.cc:504] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.190321 28507 leader_election.cc:288] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.190480 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc07eb64969465fb2b152337dc82851" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.190480 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc07eb64969465fb2b152337dc82851" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.190544 27451 raft_consensus.cc:2454] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.190544 27721 raft_consensus.cc:2454] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.190656 27524 leader_election.cc:302] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.190740 28507 raft_consensus.cc:2781] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.190773 28507 raft_consensus.cc:482] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.190784 28507 raft_consensus.cc:3037] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.191171 28507 raft_consensus.cc:504] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.191264 28507 leader_election.cc:288] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.191375 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc07eb64969465fb2b152337dc82851" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.191430 27451 raft_consensus.cc:3037] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.191427 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dc07eb64969465fb2b152337dc82851" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.191479 27721 raft_consensus.cc:3037] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.191934 27451 raft_consensus.cc:2454] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.191984 27721 raft_consensus.cc:2454] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.192101 27524 leader_election.cc:302] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.192217 28507 raft_consensus.cc:2781] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.192291 28507 raft_consensus.cc:686] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.192333 28507 consensus_queue.cc:237] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.194468 27194 catalog_manager.cc:5484] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.195583 27194 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ba19958baec64dc5988857e73a111865" } modify_external_catalogs: true
I20240503 02:09:15.195657 27194 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ba19958baec64dc5988857e73a111865" } modify_external_catalogs: true
I20240503 02:09:15.196238 28767 raft_consensus.cc:482] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.196293 28767 raft_consensus.cc:504] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.196442 28767 leader_election.cc:288] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.196604 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2cbd420b867b4144985103d498b1ea73
I20240503 02:09:15.196591 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63e82b82ee7649d88867dbb9eef8a9de" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.196655 27721 raft_consensus.cc:2454] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.196830 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cbd420b867b4144985103d498b1ea73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.196847 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cbd420b867b4144985103d498b1ea73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.196875 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63e82b82ee7649d88867dbb9eef8a9de" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.196935 27582 raft_consensus.cc:2454] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.196933 28661 tablet_replica.cc:330] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.196988 28661 raft_consensus.cc:2229] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.196996 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cbd420b867b4144985103d498b1ea73 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.197039 28661 raft_consensus.cc:2258] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.197068 27386 leader_election.cc:302] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.197098 28653 tablet_replica.cc:330] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.197149 28653 raft_consensus.cc:2229] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.197167 28654 tablet_replica.cc:330] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.197178 28653 raft_consensus.cc:2258] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.197230 28654 raft_consensus.cc:2229] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.197407 28661 ts_tablet_manager.cc:1875] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.197451 28654 raft_consensus.cc:2258] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.197558 28767 raft_consensus.cc:2781] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.197587 28653 ts_tablet_manager.cc:1875] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.197603 28767 raft_consensus.cc:482] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.197654 28767 raft_consensus.cc:3037] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.198105 28767 raft_consensus.cc:504] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.198225 28767 leader_election.cc:288] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.198400 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63e82b82ee7649d88867dbb9eef8a9de" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.198457 27721 raft_consensus.cc:3037] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.198770 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63e82b82ee7649d88867dbb9eef8a9de" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.198799 28661 ts_tablet_manager.cc:1888] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.198827 27582 raft_consensus.cc:3037] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.198848 28661 log.cc:1196] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2cbd420b867b4144985103d498b1ea73
I20240503 02:09:15.198886 28653 ts_tablet_manager.cc:1888] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.198925 28653 log.cc:1196] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2cbd420b867b4144985103d498b1ea73
I20240503 02:09:15.198920 27721 raft_consensus.cc:2454] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.199057 28661 ts_tablet_manager.cc:1909] T 2cbd420b867b4144985103d498b1ea73 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.199103 28653 ts_tablet_manager.cc:1909] T 2cbd420b867b4144985103d498b1ea73 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.199234 27388 leader_election.cc:302] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.199314 28767 raft_consensus.cc:2781] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.199340 27582 raft_consensus.cc:2454] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.199352 28767 raft_consensus.cc:686] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.199383 28654 ts_tablet_manager.cc:1875] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.199390 28767 consensus_queue.cc:237] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.199582 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]) successfully deleted
I20240503 02:09:15.199839 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]) successfully deleted
I20240503 02:09:15.200543 28654 ts_tablet_manager.cc:1888] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.200598 28654 log.cc:1196] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2cbd420b867b4144985103d498b1ea73
I20240503 02:09:15.200769 28654 ts_tablet_manager.cc:1909] T 2cbd420b867b4144985103d498b1ea73 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.201087 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2cbd420b867b4144985103d498b1ea73 (table default.Table_ba19958baec64dc5988857e73a111865 [id=ad081ee0810b483e873abf1fd1abf746]) successfully deleted
I20240503 02:09:15.201166 27194 catalog_manager.cc:5484] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.206264 28410 raft_consensus.cc:482] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.206315 28410 raft_consensus.cc:504] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.206427 28410 leader_election.cc:288] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.206552 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e8dd055a1d4673811f6e775340f008" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.206610 27721 raft_consensus.cc:2454] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.206665 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e8dd055a1d4673811f6e775340f008" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.206722 27451 raft_consensus.cc:2454] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.206727 27523 leader_election.cc:302] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.206806 28410 raft_consensus.cc:2781] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.206846 28410 raft_consensus.cc:482] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.206869 28410 raft_consensus.cc:3037] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.207358 28774 raft_consensus.cc:482] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.207412 28774 raft_consensus.cc:504] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.207509 28774 leader_election.cc:288] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.207556 28410 raft_consensus.cc:504] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.207641 28410 leader_election.cc:288] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.207664 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3858a2107d6f434aaad8156a4f617115" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.207722 27451 raft_consensus.cc:2454] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.207741 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3858a2107d6f434aaad8156a4f617115" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.207787 27582 raft_consensus.cc:2454] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.208128 27656 leader_election.cc:302] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.208204 28774 raft_consensus.cc:2781] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.208247 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e8dd055a1d4673811f6e775340f008" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.208250 28774 raft_consensus.cc:482] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.208323 27721 raft_consensus.cc:3037] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.208328 28774 raft_consensus.cc:3037] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.208542 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43e8dd055a1d4673811f6e775340f008" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.208599 27451 raft_consensus.cc:3037] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.208645 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet 2ad4f6e1ac014739b60a30111b64a7c1 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.208808 27721 raft_consensus.cc:2454] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.208957 28774 raft_consensus.cc:504] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.209077 28774 leader_election.cc:288] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.209089 27451 raft_consensus.cc:2454] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.209172 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3858a2107d6f434aaad8156a4f617115" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.209228 27449 raft_consensus.cc:3037] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.209403 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3858a2107d6f434aaad8156a4f617115" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.209462 27582 raft_consensus.cc:3037] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.209408 27523 leader_election.cc:302] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.209583 28410 raft_consensus.cc:2781] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.209626 28410 raft_consensus.cc:686] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.209661 28410 consensus_queue.cc:237] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.209718 27848 consensus_queue.cc:1035] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.209832 27449 raft_consensus.cc:2454] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.209851 28507 raft_consensus.cc:482] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.209892 28507 raft_consensus.cc:504] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.209908 27582 raft_consensus.cc:2454] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.209952 28507 leader_election.cc:288] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.209965 27659 leader_election.cc:302] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.210078 28774 raft_consensus.cc:2781] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.210114 28774 raft_consensus.cc:686] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.210145 28774 consensus_queue.cc:237] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.210278 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c9273bebae49509f713f02bc050cf5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.210333 27722 raft_consensus.cc:2454] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.210659 27523 leader_election.cc:302] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.210729 28507 raft_consensus.cc:2781] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.210757 28507 raft_consensus.cc:482] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.210772 28507 raft_consensus.cc:3037] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.210800 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c9273bebae49509f713f02bc050cf5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.210857 27449 raft_consensus.cc:2454] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.210955 27199 catalog_manager.cc:5484] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.211263 28507 raft_consensus.cc:504] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.211360 28507 leader_election.cc:288] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.211393 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_998ab3c38dc14137985b066bb7f5d8a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.211505 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_998ab3c38dc14137985b066bb7f5d8a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.211484 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c9273bebae49509f713f02bc050cf5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.211545 27722 raft_consensus.cc:3037] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.211755 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c9273bebae49509f713f02bc050cf5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.211807 27449 raft_consensus.cc:3037] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.212105 27722 raft_consensus.cc:2454] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.212246 27523 leader_election.cc:302] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.212333 28507 raft_consensus.cc:2781] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.212343 27449 raft_consensus.cc:2454] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.212301 27203 catalog_manager.cc:5484] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.212375 28507 raft_consensus.cc:686] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.212415 28507 consensus_queue.cc:237] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.212916 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ee715643ae554caea8c70ede6c01c912" } modify_external_catalogs: true
I20240503 02:09:15.212965 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bc4cb4cfddac46e28864e11a79981490"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.213069 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc4cb4cfddac46e28864e11a79981490 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.213200 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.213268 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.213398 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8e4e402a9cb440a5af09d52a47056a0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.213477 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e4e402a9cb440a5af09d52a47056a0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.213461 27848 consensus_queue.cc:1035] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.212978 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ee715643ae554caea8c70ede6c01c912" } modify_external_catalogs: true
I20240503 02:09:15.214761 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_679392a2422e43f88ca52faf229d218a" } modify_external_catalogs: true
I20240503 02:09:15.214814 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_679392a2422e43f88ca52faf229d218a" } modify_external_catalogs: true
I20240503 02:09:15.215818 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b8fafdd73d1746d694a46c9f0abb4266
I20240503 02:09:15.216017 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.216100 28654 tablet_replica.cc:330] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.216167 28654 raft_consensus.cc:2229] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.216171 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.216197 28654 raft_consensus.cc:2258] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.216244 28661 tablet_replica.cc:330] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.216295 28661 raft_consensus.cc:2229] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.216321 28661 raft_consensus.cc:2258] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.216385 28654 ts_tablet_manager.cc:1875] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.216629 28661 ts_tablet_manager.cc:1875] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.216759 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2ad4f6e1ac014739b60a30111b64a7c1
I20240503 02:09:15.216846 27194 catalog_manager.cc:6554] ReplaceTablet: tablet 2ad4f6e1ac014739b60a30111b64a7c1 deleted and replaced by tablet dde17e7bed0e487189e33d614ab11a9e
I20240503 02:09:15.217029 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 with delete_type TABLET_DATA_DELETED (replaced by tablet dde17e7bed0e487189e33d614ab11a9e) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.217579 28661 ts_tablet_manager.cc:1888] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.217623 28661 log.cc:1196] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b8fafdd73d1746d694a46c9f0abb4266
I20240503 02:09:15.217617 28654 ts_tablet_manager.cc:1888] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.217659 28654 log.cc:1196] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b8fafdd73d1746d694a46c9f0abb4266
I20240503 02:09:15.217777 28661 ts_tablet_manager.cc:1909] T b8fafdd73d1746d694a46c9f0abb4266 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.217813 28654 ts_tablet_manager.cc:1909] T b8fafdd73d1746d694a46c9f0abb4266 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.217893 28766 consensus_queue.cc:1035] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.218156 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.218210 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 with delete_type TABLET_DATA_DELETED (replaced by tablet dde17e7bed0e487189e33d614ab11a9e) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.218268 28661 tablet_replica.cc:330] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.218305 28661 raft_consensus.cc:2229] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.218334 28661 raft_consensus.cc:2258] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.218528 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet b8fafdd73d1746d694a46c9f0abb4266 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.218573 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ccd6dc9cfb834653ae08c29ff0f4848d
I20240503 02:09:15.218608 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ad4f6e1ac014739b60a30111b64a7c1 with delete_type TABLET_DATA_DELETED (replaced by tablet dde17e7bed0e487189e33d614ab11a9e) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.218652 28653 tablet_replica.cc:330] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.218693 28653 raft_consensus.cc:2229] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.218722 28653 raft_consensus.cc:2258] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.218859 28653 ts_tablet_manager.cc:1875] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.218981 28654 tablet_replica.cc:330] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.219048 28654 raft_consensus.cc:2229] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.219080 28654 raft_consensus.cc:2258] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.219241 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.219257 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.219285 28654 ts_tablet_manager.cc:1875] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.219918 28409 consensus_queue.cc:1035] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.219995 28653 ts_tablet_manager.cc:1888] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.220036 28653 log.cc:1196] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b8fafdd73d1746d694a46c9f0abb4266
I20240503 02:09:15.220175 28653 ts_tablet_manager.cc:1909] T b8fafdd73d1746d694a46c9f0abb4266 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.220285 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.220338 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet ccd6dc9cfb834653ae08c29ff0f4848d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.220549 28653 tablet_replica.cc:330] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.220598 28653 raft_consensus.cc:2229] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.220652 28653 raft_consensus.cc:2258] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.220753 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b8fafdd73d1746d694a46c9f0abb4266 (table default.Table_ee715643ae554caea8c70ede6c01c912 [id=45920270e4844c67b95c42a1fd6f235d]) successfully deleted
I20240503 02:09:15.220943 28661 ts_tablet_manager.cc:1875] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.221276 28654 ts_tablet_manager.cc:1888] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.221340 28654 log.cc:1196] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2ad4f6e1ac014739b60a30111b64a7c1
I20240503 02:09:15.221478 28653 ts_tablet_manager.cc:1875] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.221524 28654 ts_tablet_manager.cc:1909] T 2ad4f6e1ac014739b60a30111b64a7c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.222060 28654 tablet_replica.cc:330] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.222076 28661 ts_tablet_manager.cc:1888] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.222113 28661 log.cc:1196] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2ad4f6e1ac014739b60a30111b64a7c1
I20240503 02:09:15.222118 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2ad4f6e1ac014739b60a30111b64a7c1 (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:15.222256 28654 raft_consensus.cc:2229] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.222280 28661 ts_tablet_manager.cc:1909] T 2ad4f6e1ac014739b60a30111b64a7c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.222298 28654 raft_consensus.cc:2258] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.222484 28654 ts_tablet_manager.cc:1875] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.222731 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 78311555b20a42d9b2884445a9dcf925 (DEFAULT_TABLE table=default.Table_998ab3c38dc14137985b066bb7f5d8a4 [id=37de95f92d3c41d59eb50ebe5d4652c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.222779 28653 ts_tablet_manager.cc:1888] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.222819 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78311555b20a42d9b2884445a9dcf925. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.223409 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 78311555b20a42d9b2884445a9dcf925 (DEFAULT_TABLE table=default.Table_998ab3c38dc14137985b066bb7f5d8a4 [id=37de95f92d3c41d59eb50ebe5d4652c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.223501 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78311555b20a42d9b2884445a9dcf925. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.223819 28654 ts_tablet_manager.cc:1888] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.223909 28655 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.224138 27690 tablet_service.cc:1467] Processing CreateTablet for tablet 78311555b20a42d9b2884445a9dcf925 (DEFAULT_TABLE table=default.Table_998ab3c38dc14137985b066bb7f5d8a4 [id=37de95f92d3c41d59eb50ebe5d4652c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.224215 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78311555b20a42d9b2884445a9dcf925. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.224254 28655 tablet_bootstrap.cc:654] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.223860 28654 log.cc:1196] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ccd6dc9cfb834653ae08c29ff0f4848d
I20240503 02:09:15.224817 28656 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.225204 28656 tablet_bootstrap.cc:654] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.225369 28664 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.225494 27848 raft_consensus.cc:482] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.225546 27848 raft_consensus.cc:504] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.225630 27848 leader_election.cc:288] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.225647 27433 tablet_service.cc:1467] Processing CreateTablet for tablet dde17e7bed0e487189e33d614ab11a9e (DEFAULT_TABLE table=default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.225669 27691 tablet_service.cc:1467] Processing CreateTablet for tablet c993bef72a984db794e4cad2aca7af72 (DEFAULT_TABLE table=default.Table_bc4cb4cfddac46e28864e11a79981490 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.225725 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde17e7bed0e487189e33d614ab11a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.225726 28664 tablet_bootstrap.cc:654] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.225752 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993bef72a984db794e4cad2aca7af72. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:15.225881 27523 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:15.226009 27194 catalog_manager.cc:5484] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.226773 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 061caedc2c0647a4ae151e296643e779 (DEFAULT_TABLE table=default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.226847 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061caedc2c0647a4ae151e296643e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.226863 27692 tablet_service.cc:1467] Processing CreateTablet for tablet dde17e7bed0e487189e33d614ab11a9e (DEFAULT_TABLE table=default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.226893 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ecb958f7920a44ff9aebead671143001" } new_table_name: "default.Table_3e4e78e5e04446d2b2287cf636ec1e5f" modify_external_catalogs: true
I20240503 02:09:15.226944 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde17e7bed0e487189e33d614ab11a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.227677 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 061caedc2c0647a4ae151e296643e779 (DEFAULT_TABLE table=default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.227741 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061caedc2c0647a4ae151e296643e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.228067 27560 tablet_service.cc:1467] Processing CreateTablet for tablet dde17e7bed0e487189e33d614ab11a9e (DEFAULT_TABLE table=default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.228107 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 061caedc2c0647a4ae151e296643e779 (DEFAULT_TABLE table=default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.228164 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dde17e7bed0e487189e33d614ab11a9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.228282 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 061caedc2c0647a4ae151e296643e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.228730 27419 tablet_service.cc:1467] Processing CreateTablet for tablet c993bef72a984db794e4cad2aca7af72 (DEFAULT_TABLE table=default.Table_bc4cb4cfddac46e28864e11a79981490 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.228796 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993bef72a984db794e4cad2aca7af72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.228086 27558 tablet_service.cc:1467] Processing CreateTablet for tablet c993bef72a984db794e4cad2aca7af72 (DEFAULT_TABLE table=default.Table_bc4cb4cfddac46e28864e11a79981490 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.229386 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993bef72a984db794e4cad2aca7af72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.229751 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 81d0a40d51684646a0c7964f2a265df9 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.229789 28656 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.229825 28656 ts_tablet_manager.cc:1367] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:15.229835 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d0a40d51684646a0c7964f2a265df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.229959 28656 raft_consensus.cc:348] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.230017 28656 raft_consensus.cc:374] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.230036 28656 raft_consensus.cc:729] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
W20240503 02:09:15.230049 27524 leader_election.cc:341] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: be7a66cc87834e739e958a2caf73de0c
I20240503 02:09:15.230098 27524 leader_election.cc:302] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:15.230165 27848 raft_consensus.cc:2726] T be7a66cc87834e739e958a2caf73de0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:15.230201 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 81d0a40d51684646a0c7964f2a265df9 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.230268 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d0a40d51684646a0c7964f2a265df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.230953 27690 tablet_service.cc:1467] Processing CreateTablet for tablet 81d0a40d51684646a0c7964f2a265df9 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.231055 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d0a40d51684646a0c7964f2a265df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.231254 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_185b871b5e1049c28c3690a576682aaa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.231281 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.231341 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_185b871b5e1049c28c3690a576682aaa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.224828 28654 ts_tablet_manager.cc:1909] T ccd6dc9cfb834653ae08c29ff0f4848d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
W20240503 02:09:15.231377 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.231487 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2e19130dfc744eea8079daaafd81a7b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.231567 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e19130dfc744eea8079daaafd81a7b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.230063 28656 consensus_queue.cc:260] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.231645 28656 ts_tablet_manager.cc:1398] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.231719 28656 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.231706 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_48d2ff81d3764b7ab1f1b66d91548de9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.231796 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48d2ff81d3764b7ab1f1b66d91548de9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.231981 28655 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.232026 28655 ts_tablet_manager.cc:1367] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:15.232026 28656 tablet_bootstrap.cc:654] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.232164 28655 raft_consensus.cc:348] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.232216 28655 raft_consensus.cc:374] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.232235 28655 raft_consensus.cc:729] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.232199 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_eb8d54bef9994f61abd815aa9dd35780"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.232208 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ccd6dc9cfb834653ae08c29ff0f4848d (table default.Table_679392a2422e43f88ca52faf229d218a [id=0e67e01150414cfda8f556738345b4be]) successfully deleted
I20240503 02:09:15.232230 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8479162a1ff24fad8162a05b6165e50b" } new_table_name: "default.Table_d400c7ed3eac45339ebf652072ddb986" modify_external_catalogs: true
W20240503 02:09:15.232298 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb8d54bef9994f61abd815aa9dd35780 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.232275 28655 consensus_queue.cc:260] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.232376 28655 ts_tablet_manager.cc:1398] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.232441 28655 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.232867 28655 tablet_bootstrap.cc:654] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.234236 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c11ab6468184080a8697375a119547c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.234776 28686 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.234983 28698 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.222822 28653 log.cc:1196] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2ad4f6e1ac014739b60a30111b64a7c1
I20240503 02:09:15.235481 28706 tablet.cc:1722] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.235558 28653 ts_tablet_manager.cc:1909] T 2ad4f6e1ac014739b60a30111b64a7c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.235738 28653 tablet_replica.cc:330] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.235790 28653 raft_consensus.cc:2229] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.235817 28653 raft_consensus.cc:2258] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.237653 28661 tablet_replica.cc:330] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.237712 28661 raft_consensus.cc:2229] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.237866 28661 raft_consensus.cc:2258] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.238013 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2ad4f6e1ac014739b60a30111b64a7c1 (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:15.238070 28661 ts_tablet_manager.cc:1875] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.238227 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9c11ab6468184080a8697375a119547c
I20240503 02:09:15.238288 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 9c11ab6468184080a8697375a119547c deleted and replaced by tablet 8825d358600d49cea73d51fa734e53ca
I20240503 02:09:15.238325 28706 tablet.cc:1722] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.238405 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c11ab6468184080a8697375a119547c with delete_type TABLET_DATA_DELETED (replaced by tablet 8825d358600d49cea73d51fa734e53ca) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.238444 28686 tablet.cc:1722] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.238607 28653 ts_tablet_manager.cc:1875] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.239248 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c11ab6468184080a8697375a119547c with delete_type TABLET_DATA_DELETED (replaced by tablet 8825d358600d49cea73d51fa734e53ca) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.239320 28698 tablet.cc:1722] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.239334 28654 tablet_replica.cc:330] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.239452 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 9448d14b53f74e7f8ccbc19bb07b3a6d from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.239470 28654 raft_consensus.cc:2229] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.239498 28654 raft_consensus.cc:2258] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.239593 28654 ts_tablet_manager.cc:1875] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.239805 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2ad4f6e1ac014739b60a30111b64a7c1 (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:15.240170 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_48d2ff81d3764b7ab1f1b66d91548de9" } new_table_name: "default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce" modify_external_catalogs: true
I20240503 02:09:15.240459 28661 ts_tablet_manager.cc:1888] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.240502 28661 log.cc:1196] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ccd6dc9cfb834653ae08c29ff0f4848d
I20240503 02:09:15.240681 28661 ts_tablet_manager.cc:1909] T ccd6dc9cfb834653ae08c29ff0f4848d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.240741 28654 ts_tablet_manager.cc:1888] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.240775 28654 log.cc:1196] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9c11ab6468184080a8697375a119547c
I20240503 02:09:15.240970 28654 ts_tablet_manager.cc:1909] T 9c11ab6468184080a8697375a119547c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.241065 28661 tablet_replica.cc:330] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.241123 28661 raft_consensus.cc:2229] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.241173 28661 raft_consensus.cc:2258] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.241250 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ccd6dc9cfb834653ae08c29ff0f4848d (table default.Table_679392a2422e43f88ca52faf229d218a [id=0e67e01150414cfda8f556738345b4be]) successfully deleted
I20240503 02:09:15.241266 28661 ts_tablet_manager.cc:1875] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.241870 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9448d14b53f74e7f8ccbc19bb07b3a6d
I20240503 02:09:15.241927 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 9448d14b53f74e7f8ccbc19bb07b3a6d deleted and replaced by tablet 327dc0c3209f454499caeb31cdbe3f5d
I20240503 02:09:15.242229 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_eb8d54bef9994f61abd815aa9dd35780" } new_table_name: "default.Table_5fdda3bf91cc41fd941eee821d4ec456" modify_external_catalogs: true
I20240503 02:09:15.242236 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d with delete_type TABLET_DATA_DELETED (replaced by tablet 327dc0c3209f454499caeb31cdbe3f5d) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.242251 27557 tablet_service.cc:1467] Processing CreateTablet for tablet c08af667e8174205a3e31c4892085fb9 (DEFAULT_TABLE table=default.Table_eb8d54bef9994f61abd815aa9dd35780 [id=784bc279fe5c41f68dc2bd9ecbbb887d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.242365 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08af667e8174205a3e31c4892085fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.242451 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c11ab6468184080a8697375a119547c with delete_type TABLET_DATA_DELETED (replaced by tablet 8825d358600d49cea73d51fa734e53ca) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.242511 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d with delete_type TABLET_DATA_DELETED (replaced by tablet 327dc0c3209f454499caeb31cdbe3f5d) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.242532 27690 tablet_service.cc:1467] Processing CreateTablet for tablet c08af667e8174205a3e31c4892085fb9 (DEFAULT_TABLE table=default.Table_eb8d54bef9994f61abd815aa9dd35780 [id=784bc279fe5c41f68dc2bd9ecbbb887d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.242583 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 11b157b6f5d04c31ac019de2d800d112 (DEFAULT_TABLE table=default.Table_2e19130dfc744eea8079daaafd81a7b0 [id=9205c1ddc30a4442aa89c0b58b5b7700]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.242687 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b157b6f5d04c31ac019de2d800d112. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.242938 28653 ts_tablet_manager.cc:1888] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.242978 28653 log.cc:1196] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ccd6dc9cfb834653ae08c29ff0f4848d
I20240503 02:09:15.243350 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1535f39a27964b7e8dc4b587fc88a2b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.243448 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.243479 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08af667e8174205a3e31c4892085fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.243597 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa99492baf64897afd77f6466ae805c (DEFAULT_TABLE table=default.Table_48d2ff81d3764b7ab1f1b66d91548de9 [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.243607 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b] alter complete (version 2)
I20240503 02:09:15.243666 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa99492baf64897afd77f6466ae805c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.244470 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 11b157b6f5d04c31ac019de2d800d112 (DEFAULT_TABLE table=default.Table_2e19130dfc744eea8079daaafd81a7b0 [id=9205c1ddc30a4442aa89c0b58b5b7700]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.244573 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b157b6f5d04c31ac019de2d800d112. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.244576 27425 tablet_service.cc:1467] Processing CreateTablet for tablet c08af667e8174205a3e31c4892085fb9 (DEFAULT_TABLE table=default.Table_eb8d54bef9994f61abd815aa9dd35780 [id=784bc279fe5c41f68dc2bd9ecbbb887d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.244657 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08af667e8174205a3e31c4892085fb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.245005 28661 ts_tablet_manager.cc:1888] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.245049 28661 log.cc:1196] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9c11ab6468184080a8697375a119547c
I20240503 02:09:15.245221 28661 ts_tablet_manager.cc:1909] T 9c11ab6468184080a8697375a119547c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.245251 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 9448d14b53f74e7f8ccbc19bb07b3a6d with delete_type TABLET_DATA_DELETED (replaced by tablet 327dc0c3209f454499caeb31cdbe3f5d) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.245414 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa99492baf64897afd77f6466ae805c (DEFAULT_TABLE table=default.Table_48d2ff81d3764b7ab1f1b66d91548de9 [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.245487 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa99492baf64897afd77f6466ae805c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.245630 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa99492baf64897afd77f6466ae805c (DEFAULT_TABLE table=default.Table_48d2ff81d3764b7ab1f1b66d91548de9 [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.245712 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa99492baf64897afd77f6466ae805c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.246026 28653 ts_tablet_manager.cc:1909] T ccd6dc9cfb834653ae08c29ff0f4848d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
W20240503 02:09:15.246179 27202 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1fa99492baf64897afd77f6466ae805c consensus state has no leader
I20240503 02:09:15.246214 28653 tablet_replica.cc:330] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.246260 28653 raft_consensus.cc:2229] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.246285 28653 raft_consensus.cc:2258] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.246398 28653 ts_tablet_manager.cc:1875] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.246423 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ccd6dc9cfb834653ae08c29ff0f4848d (table default.Table_679392a2422e43f88ca52faf229d218a [id=0e67e01150414cfda8f556738345b4be]) successfully deleted
I20240503 02:09:15.246568 27848 raft_consensus.cc:482] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.246608 27848 raft_consensus.cc:504] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.246697 27848 leader_election.cc:288] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.247026 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce6b30fc5c742579f061c4bc50ecdf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.247090 27721 raft_consensus.cc:2454] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.247161 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce6b30fc5c742579f061c4bc50ecdf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.247220 27449 raft_consensus.cc:2454] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.247287 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 11b157b6f5d04c31ac019de2d800d112 (DEFAULT_TABLE table=default.Table_2e19130dfc744eea8079daaafd81a7b0 [id=9205c1ddc30a4442aa89c0b58b5b7700]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.247356 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11b157b6f5d04c31ac019de2d800d112. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.247346 27523 leader_election.cc:302] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.247417 27848 raft_consensus.cc:2781] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.247445 27848 raft_consensus.cc:482] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.247462 27848 raft_consensus.cc:3037] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.247566 28661 tablet_replica.cc:330] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.247611 28661 raft_consensus.cc:2229] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.247634 28661 raft_consensus.cc:2258] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.247705 28661 ts_tablet_manager.cc:1875] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.247928 28653 ts_tablet_manager.cc:1888] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.247969 28653 log.cc:1196] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9c11ab6468184080a8697375a119547c
I20240503 02:09:15.248049 27848 raft_consensus.cc:504] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.248139 28653 ts_tablet_manager.cc:1909] T 9c11ab6468184080a8697375a119547c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.248150 27848 leader_election.cc:288] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.248366 28653 tablet_replica.cc:330] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.248415 28653 raft_consensus.cc:2229] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.248407 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce6b30fc5c742579f061c4bc50ecdf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.248445 28653 raft_consensus.cc:2258] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.248463 27721 raft_consensus.cc:3037] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.248522 28653 ts_tablet_manager.cc:1875] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.248646 28654 tablet_replica.cc:330] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.248692 28654 raft_consensus.cc:2229] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.248733 28654 pending_rounds.cc:62] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:15.248749 28654 pending_rounds.cc:68] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420026404245504 op_type: NO_OP noop_request { }
I20240503 02:09:15.248777 28654 raft_consensus.cc:2866] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.248795 28654 raft_consensus.cc:2258] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.248972 28654 ts_tablet_manager.cc:1875] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.248979 27721 raft_consensus.cc:2454] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.249223 27523 leader_election.cc:302] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.249348 28655 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.249362 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9c11ab6468184080a8697375a119547c (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:15.249385 28655 ts_tablet_manager.cc:1367] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240503 02:09:15.249419 27848 raft_consensus.cc:2781] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.249462 27848 raft_consensus.cc:686] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.249456 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9c11ab6468184080a8697375a119547c (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:15.249502 28655 raft_consensus.cc:348] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.249505 27848 consensus_queue.cc:237] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.249552 28655 raft_consensus.cc:374] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.249568 28655 raft_consensus.cc:729] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.249601 28655 consensus_queue.cc:260] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.249680 28655 ts_tablet_manager.cc:1398] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.249735 28655 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.249751 28653 ts_tablet_manager.cc:1888] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.249792 28653 log.cc:1196] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9448d14b53f74e7f8ccbc19bb07b3a6d
I20240503 02:09:15.249845 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9c11ab6468184080a8697375a119547c (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:15.249943 28653 ts_tablet_manager.cc:1909] T 9448d14b53f74e7f8ccbc19bb07b3a6d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.250059 28655 tablet_bootstrap.cc:654] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.250123 28654 ts_tablet_manager.cc:1888] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.250161 28654 log.cc:1196] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9448d14b53f74e7f8ccbc19bb07b3a6d
I20240503 02:09:15.250188 28664 tablet_bootstrap.cc:492] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.250223 28664 ts_tablet_manager.cc:1367] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.001s
I20240503 02:09:15.250330 28654 ts_tablet_manager.cc:1909] T 9448d14b53f74e7f8ccbc19bb07b3a6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.250363 28664 raft_consensus.cc:348] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.250414 28664 raft_consensus.cc:374] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.250432 28664 raft_consensus.cc:729] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.250460 28664 consensus_queue.cc:260] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.250541 28664 ts_tablet_manager.cc:1398] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.250598 28664 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.250926 28664 tablet_bootstrap.cc:654] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.251101 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet f6022124cd954fcabb93d301a64837a8 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.248988 28661 ts_tablet_manager.cc:1888] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.251159 28661 log.cc:1196] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9448d14b53f74e7f8ccbc19bb07b3a6d
I20240503 02:09:15.251632 28661 ts_tablet_manager.cc:1909] T 9448d14b53f74e7f8ccbc19bb07b3a6d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.251665 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce6b30fc5c742579f061c4bc50ecdf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.251719 27449 raft_consensus.cc:3037] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:15.252061 27200 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c08af667e8174205a3e31c4892085fb9 consensus state has no leader
I20240503 02:09:15.252257 27449 raft_consensus.cc:2454] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.252308 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d400c7ed3eac45339ebf652072ddb986 [id=3fc56404e8894e06afad9ccc8d82cf89] alter complete (version 1)
I20240503 02:09:15.252671 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:15.252761 27690 tablet_service.cc:1467] Processing CreateTablet for tablet d3152be657da44e3ae7fd9501842c6ac (DEFAULT_TABLE table=default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3 [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.252836 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3152be657da44e3ae7fd9501842c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.252879 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 (DEFAULT_TABLE table=default.Table_185b871b5e1049c28c3690a576682aaa [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.252959 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b675ecf68b049edac996e4fc7b0bbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.253098 28656 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.253134 28656 ts_tablet_manager.cc:1367] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.000s
I20240503 02:09:15.253218 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f6022124cd954fcabb93d301a64837a8
I20240503 02:09:15.253252 28656 raft_consensus.cc:348] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.253284 27201 catalog_manager.cc:6554] ReplaceTablet: tablet f6022124cd954fcabb93d301a64837a8 deleted and replaced by tablet 63ec489aa11148a4a34b26750a624043
I20240503 02:09:15.253302 28656 raft_consensus.cc:374] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.253363 28656 raft_consensus.cc:729] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.252808 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.253412 28656 consensus_queue.cc:260] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.253446 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327dc0c3209f454499caeb31cdbe3f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.253511 28656 ts_tablet_manager.cc:1398] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.253559 28656 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.253684 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.253760 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327dc0c3209f454499caeb31cdbe3f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.253877 28656 tablet_bootstrap.cc:654] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.253986 27558 tablet_service.cc:1467] Processing CreateTablet for tablet d3152be657da44e3ae7fd9501842c6ac (DEFAULT_TABLE table=default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3 [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.254070 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3152be657da44e3ae7fd9501842c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.254552 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 8825d358600d49cea73d51fa734e53ca (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.254637 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8825d358600d49cea73d51fa734e53ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.255048 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 (DEFAULT_TABLE table=default.Table_185b871b5e1049c28c3690a576682aaa [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.255128 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b675ecf68b049edac996e4fc7b0bbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.255177 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c8acdbb592984df1a1daec4e8588b4ae" } modify_external_catalogs: true
I20240503 02:09:15.255224 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c8acdbb592984df1a1daec4e8588b4ae" } modify_external_catalogs: true
I20240503 02:09:15.255425 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1" } modify_external_catalogs: true
I20240503 02:09:15.255467 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1" } modify_external_catalogs: true
I20240503 02:09:15.255578 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_21661264ffe84bba9777c3f8f8d15678" } new_table_name: "default.Table_1e003d0ba7544d5b8d7899f8e4e32e11" modify_external_catalogs: true
I20240503 02:09:15.255720 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet f6022124cd954fcabb93d301a64837a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 63ec489aa11148a4a34b26750a624043) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.255805 28685 raft_consensus.cc:482] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.255841 28685 raft_consensus.cc:504] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.255928 28685 leader_election.cc:288] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.255986 28656 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.256019 28656 ts_tablet_manager.cc:1367] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.256039 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7c9c3fa8cd1495bb93818b2879ecfd4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.256098 27721 raft_consensus.cc:2454] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.256135 28656 raft_consensus.cc:348] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.256193 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7c9c3fa8cd1495bb93818b2879ecfd4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.256254 27582 raft_consensus.cc:2454] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.256366 27388 leader_election.cc:302] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.256431 28409 raft_consensus.cc:2781] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.256453 28409 raft_consensus.cc:482] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.256469 28409 raft_consensus.cc:3037] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.256505 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet f6022124cd954fcabb93d301a64837a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 63ec489aa11148a4a34b26750a624043) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.256978 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:15.257036 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet f6022124cd954fcabb93d301a64837a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 63ec489aa11148a4a34b26750a624043) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.257048 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.257146 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327dc0c3209f454499caeb31cdbe3f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.257375 28653 tablet_replica.cc:330] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.257426 28653 raft_consensus.cc:2229] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.257469 28653 pending_rounds.cc:62] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:15.257483 28653 pending_rounds.cc:68] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420026574368768 op_type: NO_OP noop_request { }
I20240503 02:09:15.257511 28653 raft_consensus.cc:2866] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.257529 28653 raft_consensus.cc:2258] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.257548 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 8825d358600d49cea73d51fa734e53ca (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.257628 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8825d358600d49cea73d51fa734e53ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.257872 28653 ts_tablet_manager.cc:1875] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.257051 28409 raft_consensus.cc:504] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.258610 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 (DEFAULT_TABLE table=default.Table_185b871b5e1049c28c3690a576682aaa [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.258651 28409 leader_election.cc:288] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.258684 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b675ecf68b049edac996e4fc7b0bbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.258810 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7c9c3fa8cd1495bb93818b2879ecfd4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.258857 27721 raft_consensus.cc:3037] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.258930 28653 ts_tablet_manager.cc:1888] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.258975 28653 log.cc:1196] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f6022124cd954fcabb93d301a64837a8
I20240503 02:09:15.259152 28653 ts_tablet_manager.cc:1909] T f6022124cd954fcabb93d301a64837a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.259356 27721 raft_consensus.cc:2454] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.259488 27388 leader_election.cc:302] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.256186 28656 raft_consensus.cc:374] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.259604 28656 raft_consensus.cc:729] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.259624 28409 raft_consensus.cc:2781] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.259662 28409 raft_consensus.cc:686] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.259647 28656 consensus_queue.cc:260] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.259696 28409 consensus_queue.cc:237] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.259756 28656 ts_tablet_manager.cc:1398] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.259824 28656 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.260144 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7c9c3fa8cd1495bb93818b2879ecfd4" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.260190 27582 raft_consensus.cc:3037] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.260241 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f6022124cd954fcabb93d301a64837a8 (table default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:15.260329 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 771a298237a74dd0a1dc5097ca794b5d (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.260404 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 771a298237a74dd0a1dc5097ca794b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.260444 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 8825d358600d49cea73d51fa734e53ca (DEFAULT_TABLE table=default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.260512 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8825d358600d49cea73d51fa734e53ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.260665 27582 raft_consensus.cc:2454] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.261042 28656 tablet_bootstrap.cc:654] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.261171 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c31a2f92fdcf40928ee9e37d49a3ac47
I20240503 02:09:15.261564 28661 tablet_replica.cc:330] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.261621 28661 raft_consensus.cc:2229] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.261818 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.261858 27433 tablet_service.cc:1467] Processing CreateTablet for tablet d3152be657da44e3ae7fd9501842c6ac (DEFAULT_TABLE table=default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3 [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.261945 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3152be657da44e3ae7fd9501842c6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.262405 27198 catalog_manager.cc:5484] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.262652 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
W20240503 02:09:15.262719 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet dde17e7bed0e487189e33d614ab11a9e consensus state has no leader
I20240503 02:09:15.263003 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 771a298237a74dd0a1dc5097ca794b5d (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.263026 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 63ec489aa11148a4a34b26750a624043 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.263100 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ec489aa11148a4a34b26750a624043. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.263113 28656 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.263141 28656 ts_tablet_manager.cc:1367] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:15.263252 28656 raft_consensus.cc:348] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.263298 28656 raft_consensus.cc:374] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.263314 28656 raft_consensus.cc:729] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.263337 28656 consensus_queue.cc:260] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.263417 28656 ts_tablet_manager.cc:1398] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.263459 28656 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.263516 28654 tablet_replica.cc:330] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.263557 28654 raft_consensus.cc:2229] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.263582 28654 raft_consensus.cc:2258] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.263729 28654 ts_tablet_manager.cc:1875] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.263753 28656 tablet_bootstrap.cc:654] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.264026 28661 raft_consensus.cc:2258] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.264232 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9448d14b53f74e7f8ccbc19bb07b3a6d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:15.264612 27197 catalog_manager.cc:5484] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.264844 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 78d80210ca3842fba4243bafc7498b2a
I20240503 02:09:15.265053 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d80210ca3842fba4243bafc7498b2a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.265094 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 2ca063a19a044e6c8466598f44316ff5 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.265154 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d80210ca3842fba4243bafc7498b2a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.262935 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.265434 28654 ts_tablet_manager.cc:1888] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.265487 28654 log.cc:1196] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f6022124cd954fcabb93d301a64837a8
I20240503 02:09:15.265516 28661 ts_tablet_manager.cc:1875] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.265543 27690 tablet_service.cc:1467] Processing CreateTablet for tablet 63ec489aa11148a4a34b26750a624043 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.265630 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ec489aa11148a4a34b26750a624043. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.265628 28654 ts_tablet_manager.cc:1909] T f6022124cd954fcabb93d301a64837a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.265435 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 771a298237a74dd0a1dc5097ca794b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.266006 28654 tablet_replica.cc:330] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.266057 28654 raft_consensus.cc:2229] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.266088 28654 raft_consensus.cc:2258] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.266245 28654 ts_tablet_manager.cc:1875] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.266510 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2ca063a19a044e6c8466598f44316ff5
I20240503 02:09:15.266582 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 2ca063a19a044e6c8466598f44316ff5 deleted and replaced by tablet f679cfe32cbc48c0b6c188b60c2b08aa
I20240503 02:09:15.266841 28410 raft_consensus.cc:482] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.266885 28410 raft_consensus.cc:504] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.266988 28410 leader_election.cc:288] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.267089 28410 raft_consensus.cc:482] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.267122 28410 raft_consensus.cc:504] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.267200 28410 leader_election.cc:288] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.267201 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d80210ca3842fba4243bafc7498b2a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.267261 27449 raft_consensus.cc:2454] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.267354 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 771a298237a74dd0a1dc5097ca794b5d (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.267354 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e3a4f847e4f2d9d386cbb5c09842c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.267410 27722 raft_consensus.cc:2454] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.267433 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 771a298237a74dd0a1dc5097ca794b5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.267531 27524 leader_election.cc:302] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.267604 28410 raft_consensus.cc:2781] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.267622 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f6022124cd954fcabb93d301a64837a8 (table default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:15.267637 28410 raft_consensus.cc:482] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.267654 28410 raft_consensus.cc:3037] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.267836 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e3a4f847e4f2d9d386cbb5c09842c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.267891 27449 raft_consensus.cc:2454] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.268136 28410 raft_consensus.cc:504] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.268230 28410 leader_election.cc:288] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.268373 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ca063a19a044e6c8466598f44316ff5 with delete_type TABLET_DATA_DELETED (replaced by tablet f679cfe32cbc48c0b6c188b60c2b08aa) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.268388 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d80210ca3842fba4243bafc7498b2a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.268438 27449 raft_consensus.cc:3037] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.268651 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ca063a19a044e6c8466598f44316ff5 with delete_type TABLET_DATA_DELETED (replaced by tablet f679cfe32cbc48c0b6c188b60c2b08aa) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.268981 27449 raft_consensus.cc:2454] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.269002 28654 ts_tablet_manager.cc:1888] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.269043 28654 log.cc:1196] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c31a2f92fdcf40928ee9e37d49a3ac47
I20240503 02:09:15.267544 27523 leader_election.cc:302] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.269148 28410 raft_consensus.cc:2781] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.269184 28410 raft_consensus.cc:482] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.269203 28410 raft_consensus.cc:3037] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.269299 28654 ts_tablet_manager.cc:1909] T c31a2f92fdcf40928ee9e37d49a3ac47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.269551 27524 leader_election.cc:302] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.269583 28654 tablet_replica.cc:330] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.269629 28654 raft_consensus.cc:2229] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.269631 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c31a2f92fdcf40928ee9e37d49a3ac47 (table default.Table_c8acdbb592984df1a1daec4e8588b4ae [id=f8d3271eebfc495f8684be5e39d3fb89]) successfully deleted
I20240503 02:09:15.269655 28654 raft_consensus.cc:2258] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.269680 27690 tablet_service.cc:1467] Processing CreateTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.269698 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d80210ca3842fba4243bafc7498b2a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.269743 28410 raft_consensus.cc:504] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.269749 27722 raft_consensus.cc:3037] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.269843 28410 leader_election.cc:288] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.269850 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78d80210ca3842fba4243bafc7498b2a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.269953 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e3a4f847e4f2d9d386cbb5c09842c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.269999 27449 raft_consensus.cc:3037] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.270098 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e3a4f847e4f2d9d386cbb5c09842c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.270141 27721 raft_consensus.cc:3037] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.270193 28661 ts_tablet_manager.cc:1888] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.270237 28661 log.cc:1196] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f6022124cd954fcabb93d301a64837a8
I20240503 02:09:15.270328 27722 raft_consensus.cc:2454] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.270371 28661 ts_tablet_manager.cc:1909] T f6022124cd954fcabb93d301a64837a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.269753 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f679cfe32cbc48c0b6c188b60c2b08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.270535 27449 raft_consensus.cc:2454] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.270619 27721 raft_consensus.cc:2454] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.270746 27524 leader_election.cc:302] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.270812 28410 raft_consensus.cc:2781] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.270854 28410 raft_consensus.cc:686] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.270887 28410 consensus_queue.cc:237] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.270977 27422 tablet_service.cc:1467] Processing CreateTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.271044 28664 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.271052 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f679cfe32cbc48c0b6c188b60c2b08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.271078 28664 ts_tablet_manager.cc:1367] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20240503 02:09:15.271201 28664 raft_consensus.cc:348] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.271248 28664 raft_consensus.cc:374] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.271262 28664 raft_consensus.cc:729] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.271297 28664 consensus_queue.cc:260] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.271368 28664 ts_tablet_manager.cc:1398] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.271435 28664 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.271669 28655 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.271708 28655 ts_tablet_manager.cc:1367] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:15.271744 28664 tablet_bootstrap.cc:654] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.271836 28655 raft_consensus.cc:348] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.271885 28654 ts_tablet_manager.cc:1875] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.271910 28655 raft_consensus.cc:374] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.271930 28655 raft_consensus.cc:729] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.271960 28655 consensus_queue.cc:260] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.272049 28655 ts_tablet_manager.cc:1398] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.272109 28655 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.272130 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ca063a19a044e6c8466598f44316ff5 with delete_type TABLET_DATA_DELETED (replaced by tablet f679cfe32cbc48c0b6c188b60c2b08aa) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.272442 28655 tablet_bootstrap.cc:654] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.272840 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 78d80210ca3842fba4243bafc7498b2a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.273290 27196 catalog_manager.cc:5484] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.273396 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.273741 28653 tablet_replica.cc:330] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.273794 28653 raft_consensus.cc:2229] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.273842 28653 raft_consensus.cc:2258] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.273941 28653 ts_tablet_manager.cc:1875] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.274422 28654 ts_tablet_manager.cc:1888] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.274667 28654 log.cc:1196] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/78d80210ca3842fba4243bafc7498b2a
I20240503 02:09:15.274849 28654 ts_tablet_manager.cc:1909] T 78d80210ca3842fba4243bafc7498b2a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.275005 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet c31a2f92fdcf40928ee9e37d49a3ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.274991 28653 ts_tablet_manager.cc:1888] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.275141 28653 log.cc:1196] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2ca063a19a044e6c8466598f44316ff5
I20240503 02:09:15.275149 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.275249 28654 tablet_replica.cc:330] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.275321 28653 ts_tablet_manager.cc:1909] T 2ca063a19a044e6c8466598f44316ff5 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.275508 28653 tablet_replica.cc:330] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.275568 28653 raft_consensus.cc:2229] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.275591 28653 raft_consensus.cc:2258] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.275585 28654 raft_consensus.cc:2229] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.275629 28654 raft_consensus.cc:2258] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.275707 28654 ts_tablet_manager.cc:1875] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.275811 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2ca063a19a044e6c8466598f44316ff5 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:15.276028 28653 ts_tablet_manager.cc:1875] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.276976 28654 ts_tablet_manager.cc:1888] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.276988 28656 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.277009 28654 log.cc:1196] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2ca063a19a044e6c8466598f44316ff5
I20240503 02:09:15.277027 28656 ts_tablet_manager.cc:1367] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:15.277117 28653 ts_tablet_manager.cc:1888] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.277132 28654 ts_tablet_manager.cc:1909] T 2ca063a19a044e6c8466598f44316ff5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.277153 28653 log.cc:1196] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/78d80210ca3842fba4243bafc7498b2a
I20240503 02:09:15.277149 28656 raft_consensus.cc:348] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.277201 28656 raft_consensus.cc:374] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.277222 28656 raft_consensus.cc:729] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.277266 28656 consensus_queue.cc:260] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.277295 28653 ts_tablet_manager.cc:1909] T 78d80210ca3842fba4243bafc7498b2a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.277411 28656 ts_tablet_manager.cc:1398] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.277462 28656 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.277585 28653 tablet_replica.cc:330] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.277634 28653 raft_consensus.cc:2229] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.277662 28653 raft_consensus.cc:2258] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.277664 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.277882 28653 ts_tablet_manager.cc:1875] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.278116 28656 tablet_bootstrap.cc:654] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.274991 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 63ec489aa11148a4a34b26750a624043 (DEFAULT_TABLE table=default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.278472 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63ec489aa11148a4a34b26750a624043. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.278753 28656 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.278789 28656 ts_tablet_manager.cc:1367] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.278949 28656 raft_consensus.cc:348] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.278999 28656 raft_consensus.cc:374] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.278993 28653 ts_tablet_manager.cc:1888] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.279017 28656 raft_consensus.cc:729] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.279027 28653 log.cc:1196] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c31a2f92fdcf40928ee9e37d49a3ac47
I20240503 02:09:15.279067 28656 consensus_queue.cc:260] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.279150 28653 ts_tablet_manager.cc:1909] T c31a2f92fdcf40928ee9e37d49a3ac47 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.279203 28656 ts_tablet_manager.cc:1398] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.279251 28656 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.279740 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c31a2f92fdcf40928ee9e37d49a3ac47 (table default.Table_c8acdbb592984df1a1daec4e8588b4ae [id=f8d3271eebfc495f8684be5e39d3fb89]) successfully deleted
I20240503 02:09:15.279978 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.280083 27848 consensus_queue.cc:1035] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.280226 27560 tablet_service.cc:1467] Processing CreateTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa (DEFAULT_TABLE table=default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.280324 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f679cfe32cbc48c0b6c188b60c2b08aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.280229 28656 tablet_bootstrap.cc:654] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.281534 28686 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.282716 28698 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.282934 28706 tablet.cc:1722] T 2c7dd4d111794cf2a1fc7b65acd55fe3 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.283598 28661 tablet_replica.cc:330] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.282717 27848 consensus_queue.cc:1035] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:09:15.283650 28661 raft_consensus.cc:2229] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.283968 28661 raft_consensus.cc:2258] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.284282 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f6022124cd954fcabb93d301a64837a8 (table default.Table_91792da8cc4444c1b2910bf6dda7b4fb [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:15.284358 28664 tablet_bootstrap.cc:492] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.284397 28664 ts_tablet_manager.cc:1367] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:15.284431 28655 tablet_bootstrap.cc:492] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.284473 28655 ts_tablet_manager.cc:1367] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:15.284514 28664 raft_consensus.cc:348] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.284570 28664 raft_consensus.cc:374] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.284591 28664 raft_consensus.cc:729] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.284615 28664 consensus_queue.cc:260] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.284600 28655 raft_consensus.cc:348] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.284654 28655 raft_consensus.cc:374] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.284673 28655 raft_consensus.cc:729] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.284698 28664 ts_tablet_manager.cc:1398] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.284704 28655 consensus_queue.cc:260] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.284745 28664 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.284780 28655 ts_tablet_manager.cc:1398] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.284832 28655 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.285202 28664 tablet_bootstrap.cc:654] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.285261 28655 tablet_bootstrap.cc:654] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.285632 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_1e003d0ba7544d5b8d7899f8e4e32e11 [id=352a12ca13d94899afacb8e670b99d00] alter complete (version 2)
I20240503 02:09:15.286372 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2ca063a19a044e6c8466598f44316ff5 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
W20240503 02:09:15.286527 27196 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:15.286723 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.286950 28661 ts_tablet_manager.cc:1875] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.287081 28655 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.287127 28655 ts_tablet_manager.cc:1367] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.287274 28664 tablet_bootstrap.cc:492] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.287258 28655 raft_consensus.cc:348] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.287307 28655 raft_consensus.cc:374] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.287314 28664 ts_tablet_manager.cc:1367] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.287324 28655 raft_consensus.cc:729] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.287374 28655 consensus_queue.cc:260] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.287420 28664 raft_consensus.cc:348] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.287451 28655 ts_tablet_manager.cc:1398] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.287467 28664 raft_consensus.cc:374] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.287483 28664 raft_consensus.cc:729] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.287499 28655 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.287541 28664 consensus_queue.cc:260] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.287631 28664 ts_tablet_manager.cc:1398] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.287703 28664 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.287868 28655 tablet_bootstrap.cc:654] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.288034 28664 tablet_bootstrap.cc:654] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.288391 28655 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.288439 28655 ts_tablet_manager.cc:1367] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.288568 28655 raft_consensus.cc:348] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.288628 28655 raft_consensus.cc:374] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.288645 28655 raft_consensus.cc:729] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.288683 28655 consensus_queue.cc:260] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.288772 28655 ts_tablet_manager.cc:1398] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.288825 28655 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.288892 28664 tablet_bootstrap.cc:492] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.288926 28664 ts_tablet_manager.cc:1367] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.289045 28664 raft_consensus.cc:348] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.289098 28664 raft_consensus.cc:374] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.289115 28664 raft_consensus.cc:729] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.289151 28655 tablet_bootstrap.cc:654] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.289157 28664 consensus_queue.cc:260] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.289245 28664 ts_tablet_manager.cc:1398] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.289299 28664 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.289678 28664 tablet_bootstrap.cc:654] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.290295 28656 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.290140 28661 ts_tablet_manager.cc:1888] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.290338 28656 ts_tablet_manager.cc:1367] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:15.290354 28655 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.290388 28655 ts_tablet_manager.cc:1367] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.290412 28661 log.cc:1196] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/78d80210ca3842fba4243bafc7498b2a
I20240503 02:09:15.290450 28656 raft_consensus.cc:348] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.290511 28656 raft_consensus.cc:374] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.290529 28656 raft_consensus.cc:729] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.290518 28655 raft_consensus.cc:348] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.290567 28655 raft_consensus.cc:374] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.290580 28661 ts_tablet_manager.cc:1909] T 78d80210ca3842fba4243bafc7498b2a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.290565 28656 consensus_queue.cc:260] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.290586 28655 raft_consensus.cc:729] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.290630 28655 consensus_queue.cc:260] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.290660 28656 ts_tablet_manager.cc:1398] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.290707 28656 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.290717 28655 ts_tablet_manager.cc:1398] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.290763 28655 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.290845 28661 tablet_replica.cc:330] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.290894 28661 raft_consensus.cc:2229] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.290946 28661 raft_consensus.cc:2258] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.290973 28656 tablet_bootstrap.cc:654] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.291071 28655 tablet_bootstrap.cc:654] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.291116 28661 ts_tablet_manager.cc:1875] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.291253 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 78d80210ca3842fba4243bafc7498b2a (table default.Table_550e6e10bcdb46c191f8b7dd9c30ddf1 [id=8a300650ad514265b389e710d1ee3446]) successfully deleted
I20240503 02:09:15.291399 28656 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.291432 28656 ts_tablet_manager.cc:1367] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.291553 28656 raft_consensus.cc:348] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.291601 28656 raft_consensus.cc:374] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.291616 28656 raft_consensus.cc:729] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.291658 28656 consensus_queue.cc:260] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.291731 28656 ts_tablet_manager.cc:1398] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.291934 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.292094 28656 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.292294 28664 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.292335 28664 ts_tablet_manager.cc:1367] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.292433 28656 tablet_bootstrap.cc:654] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.292454 28664 raft_consensus.cc:348] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.292527 28664 raft_consensus.cc:374] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.292547 28664 raft_consensus.cc:729] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.292582 28664 consensus_queue.cc:260] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.292600 28661 ts_tablet_manager.cc:1888] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.292650 28664 ts_tablet_manager.cc:1398] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.292717 28664 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.293036 28664 tablet_bootstrap.cc:654] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.293252 28693 consensus_queue.cc:1035] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.293728 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.293867 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.292654 28661 log.cc:1196] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c31a2f92fdcf40928ee9e37d49a3ac47
I20240503 02:09:15.294808 28664 tablet_bootstrap.cc:492] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.294859 28664 ts_tablet_manager.cc:1367] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.294953 28661 ts_tablet_manager.cc:1909] T c31a2f92fdcf40928ee9e37d49a3ac47 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.294981 28664 raft_consensus.cc:348] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.295042 28664 raft_consensus.cc:374] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.295066 28664 raft_consensus.cc:729] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.295097 28664 consensus_queue.cc:260] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.295181 28664 ts_tablet_manager.cc:1398] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.295291 28664 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.295610 28664 tablet_bootstrap.cc:654] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.295610 28661 tablet_replica.cc:330] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.295818 28661 raft_consensus.cc:2229] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.295890 28661 raft_consensus.cc:2258] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.296023 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c31a2f92fdcf40928ee9e37d49a3ac47 (table default.Table_c8acdbb592984df1a1daec4e8588b4ae [id=f8d3271eebfc495f8684be5e39d3fb89]) successfully deleted
I20240503 02:09:15.296150 28661 ts_tablet_manager.cc:1875] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.296321 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.296638 28655 tablet_bootstrap.cc:492] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.296691 28655 ts_tablet_manager.cc:1367] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:15.296890 28774 consensus_queue.cc:1035] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.297034 28655 raft_consensus.cc:348] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.297098 28655 raft_consensus.cc:374] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.297117 28655 raft_consensus.cc:729] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.297202 28655 consensus_queue.cc:260] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.297359 28655 ts_tablet_manager.cc:1398] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.297752 28655 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.298080 28655 tablet_bootstrap.cc:654] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.298843 28664 tablet_bootstrap.cc:492] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.298890 28664 ts_tablet_manager.cc:1367] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.298928 28655 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.298974 28655 ts_tablet_manager.cc:1367] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.299018 28664 raft_consensus.cc:348] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.299065 28664 raft_consensus.cc:374] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.299082 28664 raft_consensus.cc:729] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.299113 28664 consensus_queue.cc:260] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.299099 28655 raft_consensus.cc:348] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.299151 28655 raft_consensus.cc:374] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.299171 28655 raft_consensus.cc:729] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.299213 28664 ts_tablet_manager.cc:1398] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.299207 28655 consensus_queue.cc:260] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.299263 28664 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.299294 28655 ts_tablet_manager.cc:1398] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.299355 28655 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.299583 28664 tablet_bootstrap.cc:654] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.299721 28655 tablet_bootstrap.cc:654] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.299787 28774 consensus_queue.cc:1035] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.300125 28664 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.300160 28664 ts_tablet_manager.cc:1367] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.300298 28664 raft_consensus.cc:348] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.300344 28664 raft_consensus.cc:374] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.300359 28664 raft_consensus.cc:729] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.300385 28664 consensus_queue.cc:260] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.300457 28664 ts_tablet_manager.cc:1398] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.300513 28664 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.300817 28664 tablet_bootstrap.cc:654] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.301107 27853 consensus_queue.cc:1035] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.301359 28656 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.301404 28656 ts_tablet_manager.cc:1367] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:15.301616 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.301733 28656 raft_consensus.cc:348] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.301787 28656 raft_consensus.cc:374] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.301810 28656 raft_consensus.cc:729] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.301841 28656 consensus_queue.cc:260] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.301913 28656 ts_tablet_manager.cc:1398] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.301954 28656 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.302340 28656 tablet_bootstrap.cc:654] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.303102 28656 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.303141 28656 ts_tablet_manager.cc:1367] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.303258 28656 raft_consensus.cc:348] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.303313 28656 raft_consensus.cc:374] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.303333 28656 raft_consensus.cc:729] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.303366 28656 consensus_queue.cc:260] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.303457 28656 ts_tablet_manager.cc:1398] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.303516 28656 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.303828 28656 tablet_bootstrap.cc:654] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.304222 28664 tablet_bootstrap.cc:492] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.304270 28664 ts_tablet_manager.cc:1367] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.304394 28664 raft_consensus.cc:348] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.304437 28664 raft_consensus.cc:374] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.304448 28664 raft_consensus.cc:729] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.304481 28664 consensus_queue.cc:260] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.304574 28664 ts_tablet_manager.cc:1398] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.304630 28664 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.304497 28661 ts_tablet_manager.cc:1888] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.305001 28664 tablet_bootstrap.cc:654] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.305783 28661 log.cc:1196] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2ca063a19a044e6c8466598f44316ff5
I20240503 02:09:15.306105 28655 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.306159 28655 ts_tablet_manager.cc:1367] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:15.306210 28664 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.306255 28664 ts_tablet_manager.cc:1367] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.306286 28655 raft_consensus.cc:348] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.306360 28655 raft_consensus.cc:374] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.306378 28655 raft_consensus.cc:729] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.306368 28664 raft_consensus.cc:348] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.306391 28661 ts_tablet_manager.cc:1909] T 2ca063a19a044e6c8466598f44316ff5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.306412 28664 raft_consensus.cc:374] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.306437 28664 raft_consensus.cc:729] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.306434 28655 consensus_queue.cc:260] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.306469 28664 consensus_queue.cc:260] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.306550 28664 ts_tablet_manager.cc:1398] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.306563 28655 ts_tablet_manager.cc:1398] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.306609 28655 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.307010 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2ca063a19a044e6c8466598f44316ff5 (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:15.307106 28664 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.307125 28655 tablet_bootstrap.cc:654] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.307446 28664 tablet_bootstrap.cc:654] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.307827 28656 tablet_bootstrap.cc:492] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.307871 28656 ts_tablet_manager.cc:1367] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.307984 28656 raft_consensus.cc:348] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.308032 28656 raft_consensus.cc:374] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.308048 28656 raft_consensus.cc:729] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.308085 28656 consensus_queue.cc:260] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.308154 28656 ts_tablet_manager.cc:1398] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.308214 28656 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.308393 28655 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.308437 28655 ts_tablet_manager.cc:1367] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.308530 28656 tablet_bootstrap.cc:654] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.308571 28655 raft_consensus.cc:348] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.308771 28655 raft_consensus.cc:374] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.308800 28655 raft_consensus.cc:729] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.308852 28655 consensus_queue.cc:260] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.309032 28655 ts_tablet_manager.cc:1398] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.309450 28656 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.309247 28655 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.309489 28656 ts_tablet_manager.cc:1367] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.309631 28656 raft_consensus.cc:348] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.309677 28656 raft_consensus.cc:374] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.309695 28656 raft_consensus.cc:729] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.309725 28656 consensus_queue.cc:260] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.309801 28656 ts_tablet_manager.cc:1398] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.309845 28656 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.309870 28655 tablet_bootstrap.cc:654] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.310142 28656 tablet_bootstrap.cc:654] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.311117 28664 tablet_bootstrap.cc:492] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.311165 28664 ts_tablet_manager.cc:1367] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.311316 28664 raft_consensus.cc:348] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.311375 28664 raft_consensus.cc:374] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.311400 28664 raft_consensus.cc:729] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.311419 28656 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.311436 28664 consensus_queue.cc:260] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.311470 28656 ts_tablet_manager.cc:1367] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.311532 28664 ts_tablet_manager.cc:1398] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.311586 28664 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.311581 28656 raft_consensus.cc:348] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.311623 28656 raft_consensus.cc:374] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.311641 28656 raft_consensus.cc:729] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.311664 28656 consensus_queue.cc:260] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.311755 28656 ts_tablet_manager.cc:1398] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.311908 28656 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.311908 28766 raft_consensus.cc:482] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.311920 28664 tablet_bootstrap.cc:654] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.311947 28766 raft_consensus.cc:504] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.312062 28766 leader_election.cc:288] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.312187 28655 tablet_bootstrap.cc:492] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.312224 28655 ts_tablet_manager.cc:1367] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:15.312248 28656 tablet_bootstrap.cc:654] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.312283 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993bef72a984db794e4cad2aca7af72" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.312341 27722 raft_consensus.cc:2454] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.312350 28655 raft_consensus.cc:348] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.312399 28655 raft_consensus.cc:374] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.312417 28655 raft_consensus.cc:729] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.312443 28655 consensus_queue.cc:260] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.312454 27388 leader_election.cc:302] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.312530 28655 ts_tablet_manager.cc:1398] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.312523 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993bef72a984db794e4cad2aca7af72" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.312552 28685 raft_consensus.cc:2781] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.312577 27582 raft_consensus.cc:2454] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.312578 28685 raft_consensus.cc:482] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.312614 28685 raft_consensus.cc:3037] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.313186 28664 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.313239 28664 ts_tablet_manager.cc:1367] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.313249 28685 raft_consensus.cc:504] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.313385 28664 raft_consensus.cc:348] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.313441 28664 raft_consensus.cc:374] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.313459 28664 raft_consensus.cc:729] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.313494 28664 consensus_queue.cc:260] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.313580 28664 ts_tablet_manager.cc:1398] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.313624 28664 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.313928 28664 tablet_bootstrap.cc:654] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.314076 28685 leader_election.cc:288] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.314252 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993bef72a984db794e4cad2aca7af72" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.314311 27582 raft_consensus.cc:3037] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.314332 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993bef72a984db794e4cad2aca7af72" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.314376 27722 raft_consensus.cc:3037] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.314503 28507 raft_consensus.cc:482] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.314543 28685 raft_consensus.cc:482] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.314554 28507 raft_consensus.cc:504] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.314579 28685 raft_consensus.cc:504] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.314651 28507 leader_election.cc:288] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.314656 28685 leader_election.cc:288] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.314749 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b157b6f5d04c31ac019de2d800d112" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.312577 28655 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.314805 27721 raft_consensus.cc:2454] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.314786 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061caedc2c0647a4ae151e296643e779" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.314836 27587 raft_consensus.cc:2454] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.314872 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061caedc2c0647a4ae151e296643e779" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.314908 27723 raft_consensus.cc:2454] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.314939 27523 leader_election.cc:302] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.314934 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b157b6f5d04c31ac019de2d800d112" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.314985 27722 raft_consensus.cc:2454] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.314985 27449 raft_consensus.cc:2454] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.314996 28507 raft_consensus.cc:2781] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.315019 28507 raft_consensus.cc:482] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.315035 28507 raft_consensus.cc:3037] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.315112 27386 leader_election.cc:302] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.315270 27388 leader_election.cc:302] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.315358 28766 raft_consensus.cc:2781] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.315387 28766 raft_consensus.cc:482] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.315393 28664 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.315404 28766 raft_consensus.cc:3037] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.315428 28664 ts_tablet_manager.cc:1367] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.315547 28664 raft_consensus.cc:348] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.315603 28664 raft_consensus.cc:374] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.315618 28664 raft_consensus.cc:729] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.315644 28664 consensus_queue.cc:260] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.315716 28664 ts_tablet_manager.cc:1398] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.315758 28664 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.316010 28766 raft_consensus.cc:504] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.316064 28664 tablet_bootstrap.cc:654] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.316123 28766 leader_election.cc:288] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.316162 28766 raft_consensus.cc:2781] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.316191 28766 raft_consensus.cc:686] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.316226 28766 consensus_queue.cc:237] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.316224 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061caedc2c0647a4ae151e296643e779" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.316279 27587 raft_consensus.cc:3037] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.316807 27197 catalog_manager.cc:5484] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.316896 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "061caedc2c0647a4ae151e296643e779" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.316951 27722 raft_consensus.cc:3037] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.317265 28655 tablet_bootstrap.cc:654] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.317489 27722 raft_consensus.cc:2454] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.317626 27388 leader_election.cc:302] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.317764 28685 raft_consensus.cc:2781] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.317812 28685 raft_consensus.cc:686] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.317850 28685 consensus_queue.cc:237] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.318302 28664 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.318346 28664 ts_tablet_manager.cc:1367] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:15.318465 28664 raft_consensus.cc:348] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.318526 28664 raft_consensus.cc:374] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.318545 28664 raft_consensus.cc:729] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.318584 28664 consensus_queue.cc:260] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.318652 28664 ts_tablet_manager.cc:1398] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.319293 28655 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.319346 28655 ts_tablet_manager.cc:1367] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:15.319361 27197 catalog_manager.cc:5484] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.319468 28655 raft_consensus.cc:348] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.320245 28655 raft_consensus.cc:374] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.320287 28655 raft_consensus.cc:729] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.320463 28655 consensus_queue.cc:260] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.320824 28655 ts_tablet_manager.cc:1398] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.320919 28655 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.321435 28655 tablet_bootstrap.cc:654] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.322052 28655 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.322089 28655 ts_tablet_manager.cc:1367] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.322197 28655 raft_consensus.cc:348] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.322240 28655 raft_consensus.cc:374] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.322257 28655 raft_consensus.cc:729] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.322295 28655 consensus_queue.cc:260] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.322378 28655 ts_tablet_manager.cc:1398] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.322465 28655 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.323004 28655 tablet_bootstrap.cc:654] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.323488 28655 tablet_bootstrap.cc:492] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.323529 28655 ts_tablet_manager.cc:1367] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.323643 28655 raft_consensus.cc:348] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.323685 28655 raft_consensus.cc:374] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.323702 28655 raft_consensus.cc:729] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.323740 28655 consensus_queue.cc:260] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.323845 28655 ts_tablet_manager.cc:1398] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.325484 28656 tablet_bootstrap.cc:492] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.325529 28656 ts_tablet_manager.cc:1367] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:15.325672 28656 raft_consensus.cc:348] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.325724 28656 raft_consensus.cc:374] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.325742 28656 raft_consensus.cc:729] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.325778 28656 consensus_queue.cc:260] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.325897 28656 ts_tablet_manager.cc:1398] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.325968 28656 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.326037 27587 raft_consensus.cc:2454] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.325971 27582 raft_consensus.cc:2454] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.326316 28656 tablet_bootstrap.cc:654] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.326535 28685 raft_consensus.cc:482] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.326576 28685 raft_consensus.cc:504] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.326673 28685 leader_election.cc:288] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.326769 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08af667e8174205a3e31c4892085fb9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.326829 27722 raft_consensus.cc:2454] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.326843 28656 tablet_bootstrap.cc:492] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.326876 28656 ts_tablet_manager.cc:1367] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.326925 27388 leader_election.cc:302] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.326972 28656 raft_consensus.cc:348] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.327008 28685 raft_consensus.cc:2781] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.327028 28656 raft_consensus.cc:374] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.327037 28685 raft_consensus.cc:482] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.327045 28656 raft_consensus.cc:729] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.327057 28685 raft_consensus.cc:3037] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.327073 28656 consensus_queue.cc:260] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.327148 28656 ts_tablet_manager.cc:1398] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.327160 28507 raft_consensus.cc:504] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.327265 28507 leader_election.cc:288] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.327366 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b157b6f5d04c31ac019de2d800d112" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.327425 27722 raft_consensus.cc:3037] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.327647 28685 raft_consensus.cc:504] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.327767 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11b157b6f5d04c31ac019de2d800d112" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.327855 27449 raft_consensus.cc:3037] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.327903 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08af667e8174205a3e31c4892085fb9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.327957 27723 raft_consensus.cc:3037] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.328002 27722 raft_consensus.cc:2454] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.328047 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08af667e8174205a3e31c4892085fb9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.328095 27582 raft_consensus.cc:2454] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.327776 28685 leader_election.cc:288] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.328450 27449 raft_consensus.cc:2454] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.328583 27723 raft_consensus.cc:2454] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.328670 27524 leader_election.cc:302] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.328760 27848 raft_consensus.cc:2781] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.328797 27848 raft_consensus.cc:686] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.328799 27388 leader_election.cc:302] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.328831 27848 consensus_queue.cc:237] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.328871 28685 raft_consensus.cc:2781] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.328909 28685 raft_consensus.cc:686] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.328941 28685 consensus_queue.cc:237] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.328991 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08af667e8174205a3e31c4892085fb9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.329041 27582 raft_consensus.cc:3037] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.329624 27582 raft_consensus.cc:2454] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.329905 27197 catalog_manager.cc:5484] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.329977 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.330468 27196 catalog_manager.cc:5484] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.338624 27582 raft_consensus.cc:1261] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.338646 27723 raft_consensus.cc:1261] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.338799 28685 consensus_queue.cc:1035] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.338876 28409 consensus_queue.cc:1035] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.339871 28686 tablet.cc:1722] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.340145 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.340601 28706 tablet.cc:1722] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.340778 28766 consensus_queue.cc:1035] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.341032 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]). Expected version 1 got 0
I20240503 02:09:15.341810 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d] alter complete (version 1)
I20240503 02:09:15.342252 27853 consensus_queue.cc:1035] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.344684 28698 tablet.cc:1722] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.345206 28409 consensus_queue.cc:1035] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.345535 28740 consensus_queue.cc:1035] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.374130 27848 consensus_queue.cc:1035] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.375360 28507 consensus_queue.cc:1035] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.379540 28507 raft_consensus.cc:482] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.379590 28507 raft_consensus.cc:504] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.379693 28507 leader_election.cc:288] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.379814 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f679cfe32cbc48c0b6c188b60c2b08aa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.379856 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f679cfe32cbc48c0b6c188b60c2b08aa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.379890 27449 raft_consensus.cc:2454] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.379913 27723 raft_consensus.cc:2454] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.380018 27524 leader_election.cc:302] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.380110 28507 raft_consensus.cc:2781] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.380146 28507 raft_consensus.cc:482] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.380158 28507 raft_consensus.cc:3037] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.380578 28507 raft_consensus.cc:504] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.380673 28507 leader_election.cc:288] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.380795 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f679cfe32cbc48c0b6c188b60c2b08aa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.380818 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f679cfe32cbc48c0b6c188b60c2b08aa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.380846 27723 raft_consensus.cc:3037] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.380867 27449 raft_consensus.cc:3037] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.381305 27723 raft_consensus.cc:2454] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.381315 27449 raft_consensus.cc:2454] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.381486 27523 leader_election.cc:302] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.381614 28507 raft_consensus.cc:2781] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.381665 28507 raft_consensus.cc:686] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.381711 28507 consensus_queue.cc:237] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.382315 27197 catalog_manager.cc:5484] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.382726 28740 raft_consensus.cc:482] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.382786 28740 raft_consensus.cc:504] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.382879 28740 leader_election.cc:288] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.383128 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ec489aa11148a4a34b26750a624043" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.383196 27587 raft_consensus.cc:2454] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.383248 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ec489aa11148a4a34b26750a624043" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.383296 27449 raft_consensus.cc:2454] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.383445 27659 leader_election.cc:302] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.383527 28740 raft_consensus.cc:2781] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.383562 28740 raft_consensus.cc:482] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.383579 28740 raft_consensus.cc:3037] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.384073 28740 raft_consensus.cc:504] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.384163 28740 leader_election.cc:288] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.384331 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ec489aa11148a4a34b26750a624043" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.384330 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63ec489aa11148a4a34b26750a624043" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.384392 27449 raft_consensus.cc:3037] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.384399 27587 raft_consensus.cc:3037] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.384826 27587 raft_consensus.cc:2454] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.384848 27449 raft_consensus.cc:2454] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.384986 27656 leader_election.cc:302] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.385087 28740 raft_consensus.cc:2781] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.385138 28740 raft_consensus.cc:686] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.385180 28740 consensus_queue.cc:237] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.385774 27197 catalog_manager.cc:5484] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.389366 28740 consensus_queue.cc:1035] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.390980 27853 consensus_queue.cc:1035] T 09079fec9eae482ca5fccbcb69b5e61d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.393500 28507 raft_consensus.cc:482] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.393548 28507 raft_consensus.cc:504] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.393646 28507 leader_election.cc:288] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.393751 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde17e7bed0e487189e33d614ab11a9e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.393800 27723 raft_consensus.cc:2454] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.393867 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde17e7bed0e487189e33d614ab11a9e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.393925 27449 raft_consensus.cc:2454] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.393922 27523 leader_election.cc:302] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.394003 28507 raft_consensus.cc:2781] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.394047 28507 raft_consensus.cc:482] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.394064 28507 raft_consensus.cc:3037] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.394464 28507 raft_consensus.cc:504] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.394560 28507 leader_election.cc:288] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.394730 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde17e7bed0e487189e33d614ab11a9e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.394752 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dde17e7bed0e487189e33d614ab11a9e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.394788 27723 raft_consensus.cc:3037] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.394793 27449 raft_consensus.cc:3037] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.395330 27723 raft_consensus.cc:2454] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.395330 27449 raft_consensus.cc:2454] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.395478 27523 leader_election.cc:302] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.395550 28507 raft_consensus.cc:2781] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.395579 28507 raft_consensus.cc:686] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.395612 28507 consensus_queue.cc:237] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.396268 27197 catalog_manager.cc:5484] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.396346 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.397339 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2170e94d17d44288964830bfd2f564e3" } modify_external_catalogs: true
I20240503 02:09:15.397406 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2170e94d17d44288964830bfd2f564e3" } modify_external_catalogs: true
I20240503 02:09:15.398299 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d3b844df0b674e038fa1fd7993033b30
I20240503 02:09:15.398514 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b844df0b674e038fa1fd7993033b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.398571 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b844df0b674e038fa1fd7993033b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.398573 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet d3b844df0b674e038fa1fd7993033b30 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.398654 28654 tablet_replica.cc:330] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.398658 28661 tablet_replica.cc:330] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.398695 28653 tablet_replica.cc:330] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.398712 28661 raft_consensus.cc:2229] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.398715 28654 raft_consensus.cc:2229] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.398749 28654 raft_consensus.cc:2258] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.398752 28653 raft_consensus.cc:2229] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.398766 28661 raft_consensus.cc:2258] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.398780 28653 raft_consensus.cc:2258] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.398959 28661 ts_tablet_manager.cc:1875] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.398960 28654 ts_tablet_manager.cc:1875] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.399057 28653 ts_tablet_manager.cc:1875] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.400048 28654 ts_tablet_manager.cc:1888] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.400080 28654 log.cc:1196] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d3b844df0b674e038fa1fd7993033b30
I20240503 02:09:15.400226 28654 ts_tablet_manager.cc:1909] T d3b844df0b674e038fa1fd7993033b30 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.400286 28661 ts_tablet_manager.cc:1888] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.400279 28653 ts_tablet_manager.cc:1888] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.400323 28661 log.cc:1196] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d3b844df0b674e038fa1fd7993033b30
I20240503 02:09:15.400324 28653 log.cc:1196] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d3b844df0b674e038fa1fd7993033b30
I20240503 02:09:15.400460 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d3b844df0b674e038fa1fd7993033b30 (table default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]) successfully deleted
I20240503 02:09:15.400503 28653 ts_tablet_manager.cc:1909] T d3b844df0b674e038fa1fd7993033b30 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.400503 28661 ts_tablet_manager.cc:1909] T d3b844df0b674e038fa1fd7993033b30 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.400957 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d3b844df0b674e038fa1fd7993033b30 (table default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]) successfully deleted
I20240503 02:09:15.401027 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d3b844df0b674e038fa1fd7993033b30 (table default.Table_2170e94d17d44288964830bfd2f564e3 [id=58b5a1ff61a2450d8eec390c4b761048]) successfully deleted
I20240503 02:09:15.401105 28507 raft_consensus.cc:482] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.401149 28507 raft_consensus.cc:504] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.401248 28507 leader_election.cc:288] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.401291 27853 raft_consensus.cc:482] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.401360 27853 raft_consensus.cc:504] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.401440 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d0a40d51684646a0c7964f2a265df9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.401463 27853 leader_election.cc:288] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.401497 27449 raft_consensus.cc:2454] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.401525 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d0a40d51684646a0c7964f2a265df9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.401561 27723 raft_consensus.cc:2454] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.401571 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78311555b20a42d9b2884445a9dcf925" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.401597 27524 leader_election.cc:302] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.401615 27587 raft_consensus.cc:2454] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.401688 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78311555b20a42d9b2884445a9dcf925" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.401708 27656 leader_election.cc:302] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.401744 27449 raft_consensus.cc:2454] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.401751 28507 raft_consensus.cc:2781] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.401767 27853 raft_consensus.cc:2781] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.401778 28507 raft_consensus.cc:482] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.401790 27853 raft_consensus.cc:482] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.401796 28507 raft_consensus.cc:3037] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.401808 27853 raft_consensus.cc:3037] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.402184 28507 raft_consensus.cc:504] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.402223 27853 raft_consensus.cc:504] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.402289 28507 leader_election.cc:288] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.402309 27853 leader_election.cc:288] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.402411 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d0a40d51684646a0c7964f2a265df9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.402464 27451 raft_consensus.cc:3037] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.402523 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d0a40d51684646a0c7964f2a265df9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.402585 27723 raft_consensus.cc:3037] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.402931 27451 raft_consensus.cc:2454] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.402966 27723 raft_consensus.cc:2454] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.403005 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78311555b20a42d9b2884445a9dcf925" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.403051 27449 raft_consensus.cc:3037] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.403070 28507 raft_consensus.cc:482] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.403095 27523 leader_election.cc:302] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.403106 28507 raft_consensus.cc:504] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.403182 27848 raft_consensus.cc:2781] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.403221 28507 leader_election.cc:288] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.403239 27848 raft_consensus.cc:686] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.403177 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78311555b20a42d9b2884445a9dcf925" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.403280 27587 raft_consensus.cc:3037] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.403273 27848 consensus_queue.cc:237] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.403474 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8825d358600d49cea73d51fa734e53ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.403539 27723 raft_consensus.cc:2454] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.403589 27449 raft_consensus.cc:2454] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.403668 27523 leader_election.cc:302] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.403739 27848 raft_consensus.cc:2781] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.403769 27848 raft_consensus.cc:482] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.403787 27848 raft_consensus.cc:3037] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.403832 27587 raft_consensus.cc:2454] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.404011 27659 leader_election.cc:302] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.404045 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8825d358600d49cea73d51fa734e53ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.404078 27853 raft_consensus.cc:2781] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.404099 27449 raft_consensus.cc:2454] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.404115 27853 raft_consensus.cc:686] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.404150 27853 consensus_queue.cc:237] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.404373 27848 raft_consensus.cc:504] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.404444 27848 leader_election.cc:288] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.404848 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8825d358600d49cea73d51fa734e53ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.404891 27723 raft_consensus.cc:3037] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.404827 27198 catalog_manager.cc:5484] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.404929 27197 catalog_manager.cc:5484] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.405364 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8825d358600d49cea73d51fa734e53ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.405400 27723 raft_consensus.cc:2454] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.405426 27449 raft_consensus.cc:3037] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.405535 27523 leader_election.cc:302] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.405661 27848 raft_consensus.cc:2781] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.405704 27848 raft_consensus.cc:686] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.405735 27848 consensus_queue.cc:237] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.405936 27449 raft_consensus.cc:2454] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.407049 27197 catalog_manager.cc:5484] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.415922 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.416025 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.416424 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_72b3b65538eb417192d7f431d42c12cd" } modify_external_catalogs: true
I20240503 02:09:15.416394 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_57f253fedf2c4938996c82cadded8531"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.416476 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_72b3b65538eb417192d7f431d42c12cd" } modify_external_catalogs: true
W20240503 02:09:15.416575 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57f253fedf2c4938996c82cadded8531 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.416482 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_54e8f67133a5407b827dee1b3c2e0c23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.416661 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54e8f67133a5407b827dee1b3c2e0c23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.416527 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bd6940d56b554a6096a02a7ac3fb991f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.416754 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd6940d56b554a6096a02a7ac3fb991f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.418283 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c0b8f82c500549d29c0d1a41d8e111cf
I20240503 02:09:15.418480 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.418560 28661 tablet_replica.cc:330] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.418607 28661 raft_consensus.cc:2229] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.418625 27690 tablet_service.cc:1467] Processing CreateTablet for tablet 3900c1017d114230bc93d16ece225e3f (DEFAULT_TABLE table=default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.418725 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3900c1017d114230bc93d16ece225e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.418644 28661 pending_rounds.cc:62] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.419528 28664 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.419551 28661 pending_rounds.cc:68] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027412688896 op_type: NO_OP noop_request { }
I20240503 02:09:15.419605 28661 raft_consensus.cc:2866] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.419627 28661 raft_consensus.cc:2258] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.419826 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.419901 28654 tablet_replica.cc:330] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.419906 28664 tablet_bootstrap.cc:654] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.419929 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 3900c1017d114230bc93d16ece225e3f (DEFAULT_TABLE table=default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.419955 28654 raft_consensus.cc:2229] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.419982 28654 raft_consensus.cc:2258] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.420015 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3900c1017d114230bc93d16ece225e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.420049 28661 ts_tablet_manager.cc:1875] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.420625 28664 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.420663 28664 ts_tablet_manager.cc:1367] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.420787 28664 raft_consensus.cc:348] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.420835 28664 raft_consensus.cc:374] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.420850 28664 raft_consensus.cc:729] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.420886 28664 consensus_queue.cc:260] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.420977 28664 ts_tablet_manager.cc:1398] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.421077 28656 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.421075 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 771a298237a74dd0a1dc5097ca794b5d from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.421104 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b8f82c500549d29c0d1a41d8e111cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.421183 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 3900c1017d114230bc93d16ece225e3f (DEFAULT_TABLE table=default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.421264 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3900c1017d114230bc93d16ece225e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.421398 28661 ts_tablet_manager.cc:1888] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.421429 28656 tablet_bootstrap.cc:654] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.421444 28661 log.cc:1196] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c0b8f82c500549d29c0d1a41d8e111cf
I20240503 02:09:15.421573 28661 ts_tablet_manager.cc:1909] T c0b8f82c500549d29c0d1a41d8e111cf P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.422101 28656 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.422137 28656 ts_tablet_manager.cc:1367] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.422266 28656 raft_consensus.cc:348] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.422314 28656 raft_consensus.cc:374] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.422330 28656 raft_consensus.cc:729] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.422367 28656 consensus_queue.cc:260] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.422392 28655 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.422443 28656 ts_tablet_manager.cc:1398] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.422678 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bf60bed921ad4219bd0b452cf484c521" } modify_external_catalogs: true
I20240503 02:09:15.422734 28655 tablet_bootstrap.cc:654] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.422736 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bf60bed921ad4219bd0b452cf484c521" } modify_external_catalogs: true
I20240503 02:09:15.423192 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c0b8f82c500549d29c0d1a41d8e111cf (table default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]) successfully deleted
I20240503 02:09:15.423426 28654 ts_tablet_manager.cc:1875] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.423674 28655 tablet_bootstrap.cc:492] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.423704 28655 ts_tablet_manager.cc:1367] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.423799 28655 raft_consensus.cc:348] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.423837 28655 raft_consensus.cc:374] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.423848 28655 raft_consensus.cc:729] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.423888 28655 consensus_queue.cc:260] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.423967 28655 ts_tablet_manager.cc:1398] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.424633 28654 ts_tablet_manager.cc:1888] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.424674 28654 log.cc:1196] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c0b8f82c500549d29c0d1a41d8e111cf
I20240503 02:09:15.424813 28654 ts_tablet_manager.cc:1909] T c0b8f82c500549d29c0d1a41d8e111cf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.425544 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 (DEFAULT_TABLE table=default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.425617 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c08a7da01c54148bc19f3b29c00f841. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.426481 28656 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.426509 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 (DEFAULT_TABLE table=default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.426606 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c08a7da01c54148bc19f3b29c00f841. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.426857 28656 tablet_bootstrap.cc:654] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.427307 28656 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.427341 28656 ts_tablet_manager.cc:1367] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.427448 28656 raft_consensus.cc:348] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.427539 28656 raft_consensus.cc:374] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.427582 28656 raft_consensus.cc:729] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.427610 28656 consensus_queue.cc:260] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.427685 28655 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.427690 28656 ts_tablet_manager.cc:1398] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.427896 27690 tablet_service.cc:1467] Processing CreateTablet for tablet f8419275f80940f4aad5ebc259cb52fc (DEFAULT_TABLE table=default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.427981 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8419275f80940f4aad5ebc259cb52fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.428023 28655 tablet_bootstrap.cc:654] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.428115 27558 tablet_service.cc:1467] Processing CreateTablet for tablet f8419275f80940f4aad5ebc259cb52fc (DEFAULT_TABLE table=default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.428185 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8419275f80940f4aad5ebc259cb52fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.428361 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 (DEFAULT_TABLE table=default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.428440 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c08a7da01c54148bc19f3b29c00f841. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.429062 28664 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.429265 27427 tablet_service.cc:1467] Processing CreateTablet for tablet f8419275f80940f4aad5ebc259cb52fc (DEFAULT_TABLE table=default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.429361 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8419275f80940f4aad5ebc259cb52fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.429430 28664 tablet_bootstrap.cc:654] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.429919 28655 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.429955 28655 ts_tablet_manager.cc:1367] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.430073 28655 raft_consensus.cc:348] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.430116 28655 raft_consensus.cc:374] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.430132 28655 raft_consensus.cc:729] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.430179 28655 consensus_queue.cc:260] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.430239 28664 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.430269 28655 ts_tablet_manager.cc:1398] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.430277 28664 ts_tablet_manager.cc:1367] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.430382 28655 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.430385 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 388e82752def4a31b3d8bec18a478273
I20240503 02:09:15.430404 28664 raft_consensus.cc:348] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.430451 28664 raft_consensus.cc:374] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.430467 28664 raft_consensus.cc:729] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.430509 28664 consensus_queue.cc:260] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.430608 28664 ts_tablet_manager.cc:1398] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.430680 28664 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.430766 28655 tablet_bootstrap.cc:654] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.430984 28664 tablet_bootstrap.cc:654] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.431123 28653 tablet_replica.cc:330] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.431180 28653 raft_consensus.cc:2229] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.431211 28653 raft_consensus.cc:2258] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.431512 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e82752def4a31b3d8bec18a478273 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.431602 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e82752def4a31b3d8bec18a478273 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.432076 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 771a298237a74dd0a1dc5097ca794b5d
I20240503 02:09:15.432170 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 771a298237a74dd0a1dc5097ca794b5d deleted and replaced by tablet 5c89b472abbe4fceb027714afa92a7b9
I20240503 02:09:15.432914 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet 771a298237a74dd0a1dc5097ca794b5d with delete_type TABLET_DATA_DELETED (replaced by tablet 5c89b472abbe4fceb027714afa92a7b9) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.432977 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 771a298237a74dd0a1dc5097ca794b5d with delete_type TABLET_DATA_DELETED (replaced by tablet 5c89b472abbe4fceb027714afa92a7b9) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.432981 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_acc60625100f46ca9a4b6b8ee71f3a4b" } new_table_name: "default.Table_61d9e998f8ef48daaec7c5087f31e260" modify_external_catalogs: true
I20240503 02:09:15.433018 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e82752def4a31b3d8bec18a478273 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.433080 28661 tablet_replica.cc:330] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.433130 28661 raft_consensus.cc:2229] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.433183 28661 raft_consensus.cc:2258] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.433274 28661 ts_tablet_manager.cc:1875] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.433382 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet 771a298237a74dd0a1dc5097ca794b5d with delete_type TABLET_DATA_DELETED (replaced by tablet 5c89b472abbe4fceb027714afa92a7b9) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.433446 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c0b8f82c500549d29c0d1a41d8e111cf (table default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]) successfully deleted
I20240503 02:09:15.433509 28656 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.433687 28654 tablet_replica.cc:330] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.433739 28654 raft_consensus.cc:2229] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.433771 28654 raft_consensus.cc:2258] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.433856 28656 tablet_bootstrap.cc:654] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.433861 28654 ts_tablet_manager.cc:1875] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.434085 28664 tablet_bootstrap.cc:492] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.434121 28664 ts_tablet_manager.cc:1367] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.434239 28664 raft_consensus.cc:348] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.434284 28664 raft_consensus.cc:374] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.434301 28664 raft_consensus.cc:729] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.434334 28664 consensus_queue.cc:260] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.434417 28664 ts_tablet_manager.cc:1398] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.434526 28661 ts_tablet_manager.cc:1888] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.434568 28661 log.cc:1196] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/388e82752def4a31b3d8bec18a478273
I20240503 02:09:15.434736 28661 ts_tablet_manager.cc:1909] T 388e82752def4a31b3d8bec18a478273 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.435232 27690 tablet_service.cc:1467] Processing CreateTablet for tablet fd4021dcc28f40fdb505f2491e034a40 (DEFAULT_TABLE table=default.Table_54e8f67133a5407b827dee1b3c2e0c23 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.435328 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4021dcc28f40fdb505f2491e034a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.435999 28661 tablet_replica.cc:330] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.436224 28654 ts_tablet_manager.cc:1888] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.436278 28654 log.cc:1196] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/388e82752def4a31b3d8bec18a478273
I20240503 02:09:15.436468 28654 ts_tablet_manager.cc:1909] T 388e82752def4a31b3d8bec18a478273 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.435240 27853 raft_consensus.cc:482] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.436640 28654 tablet_replica.cc:330] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.436623 27853 raft_consensus.cc:504] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.436690 28654 raft_consensus.cc:2229] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.436717 28654 raft_consensus.cc:2258] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.436735 27853 leader_election.cc:288] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.436235 27558 tablet_service.cc:1467] Processing CreateTablet for tablet fd4021dcc28f40fdb505f2491e034a40 (DEFAULT_TABLE table=default.Table_54e8f67133a5407b827dee1b3c2e0c23 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.436863 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4021dcc28f40fdb505f2491e034a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.436895 28661 raft_consensus.cc:2229] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.436923 27449 raft_consensus.cc:1261] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.436941 28661 raft_consensus.cc:2258] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.436996 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "771a298237a74dd0a1dc5097ca794b5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.437053 28653 ts_tablet_manager.cc:1875] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.437072 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 388e82752def4a31b3d8bec18a478273 (table default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6]) successfully deleted
I20240503 02:09:15.437162 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 388e82752def4a31b3d8bec18a478273 (table default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6]) successfully deleted
I20240503 02:09:15.437237 27723 raft_consensus.cc:1261] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.437896 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.437924 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a543dbd910a44984910275944aa9e670"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.437975 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c89b472abbe4fceb027714afa92a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:15.438009 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a543dbd910a44984910275944aa9e670 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.437973 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0978045ecff040a09a7a1cd91e12dcb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.438066 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0978045ecff040a09a7a1cd91e12dcb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.438340 28653 ts_tablet_manager.cc:1888] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.438381 28653 log.cc:1196] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c0b8f82c500549d29c0d1a41d8e111cf
I20240503 02:09:15.438418 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_15d9a598c1ea470c8376e98dfea36c42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.438495 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15d9a598c1ea470c8376e98dfea36c42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.438529 28653 ts_tablet_manager.cc:1909] T c0b8f82c500549d29c0d1a41d8e111cf P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.438704 28654 ts_tablet_manager.cc:1875] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.439100 27848 consensus_queue.cc:1035] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.439191 27848 consensus_queue.cc:1035] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.439292 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.439363 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c89b472abbe4fceb027714afa92a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:15.439539 27656 leader_election.cc:341] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.439623 27853 raft_consensus.cc:482] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.439653 27853 raft_consensus.cc:504] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.439738 27853 leader_election.cc:288] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.439734 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f5965cead947400690406a4c61d35b7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.439822 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5965cead947400690406a4c61d35b7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.439900 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa99492baf64897afd77f6466ae805c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.439957 27451 raft_consensus.cc:2454] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.440201 27659 leader_election.cc:302] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.440352 28654 ts_tablet_manager.cc:1888] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.440744 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1a6378143d7f492196e138d6e4c73b59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.440831 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a6378143d7f492196e138d6e4c73b59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.440879 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "771a298237a74dd0a1dc5097ca794b5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.440941 27449 raft_consensus.cc:2454] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.441032 27853 raft_consensus.cc:2781] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.441066 27853 raft_consensus.cc:482] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.441064 27427 tablet_service.cc:1467] Processing CreateTablet for tablet fd4021dcc28f40fdb505f2491e034a40 (DEFAULT_TABLE table=default.Table_54e8f67133a5407b827dee1b3c2e0c23 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.441090 27853 raft_consensus.cc:3037] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.441143 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4021dcc28f40fdb505f2491e034a40. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.441458 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_49b78344ef8c4bb8a95fc6940ad9dc6c" } new_table_name: "default.Table_9beb4cf015b3468a9aa10491b78c26da" modify_external_catalogs: true
I20240503 02:09:15.441056 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa99492baf64897afd77f6466ae805c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.441629 27587 raft_consensus.cc:2454] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.441655 27659 leader_election.cc:302] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:15.441650 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 59299d27526640828eda00961363d0a6 from {username='slave'} at 127.0.0.1:39874
W20240503 02:09:15.441692 27659 raft_consensus.cc:2670] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: unable to run election callback: Service unavailable: Thread pool token was shut down
I20240503 02:09:15.441527 28654 log.cc:1196] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/771a298237a74dd0a1dc5097ca794b5d
I20240503 02:09:15.442052 28654 ts_tablet_manager.cc:1909] T 771a298237a74dd0a1dc5097ca794b5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.442359 28661 ts_tablet_manager.cc:1875] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.442551 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 771a298237a74dd0a1dc5097ca794b5d (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:15.442708 27853 raft_consensus.cc:504] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.442814 27853 leader_election.cc:288] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.442890 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 59299d27526640828eda00961363d0a6
I20240503 02:09:15.442952 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 59299d27526640828eda00961363d0a6 deleted and replaced by tablet f9f8911e604043c8a5b313ef62d206f0
I20240503 02:09:15.442960 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa99492baf64897afd77f6466ae805c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.443007 27449 raft_consensus.cc:3037] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.443461 28661 ts_tablet_manager.cc:1888] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.443501 28661 log.cc:1196] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/771a298237a74dd0a1dc5097ca794b5d
I20240503 02:09:15.443497 27449 raft_consensus.cc:2454] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.443607 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 766a17e99ae64b878920d451cc814a3c (DEFAULT_TABLE table=default.Table_0978045ecff040a09a7a1cd91e12dcb1 [id=a876b3d23f9b4ec2821cd873c06fe303]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.443643 28661 ts_tablet_manager.cc:1909] T 771a298237a74dd0a1dc5097ca794b5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.443689 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766a17e99ae64b878920d451cc814a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.443734 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 59299d27526640828eda00961363d0a6 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f8911e604043c8a5b313ef62d206f0) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.443745 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 59299d27526640828eda00961363d0a6 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f8911e604043c8a5b313ef62d206f0) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.443814 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 32d965b7253c4f7582e32802e4ec73aa (DEFAULT_TABLE table=default.Table_a543dbd910a44984910275944aa9e670 [id=a8cd12a3c7364147b9ce763639338b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.443799 28654 tablet_replica.cc:330] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.443885 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d965b7253c4f7582e32802e4ec73aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.443893 28654 raft_consensus.cc:2229] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.443925 28654 raft_consensus.cc:2258] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.444101 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 766a17e99ae64b878920d451cc814a3c (DEFAULT_TABLE table=default.Table_0978045ecff040a09a7a1cd91e12dcb1 [id=a876b3d23f9b4ec2821cd873c06fe303]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.444181 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766a17e99ae64b878920d451cc814a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.444932 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 766a17e99ae64b878920d451cc814a3c (DEFAULT_TABLE table=default.Table_0978045ecff040a09a7a1cd91e12dcb1 [id=a876b3d23f9b4ec2821cd873c06fe303]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.445025 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766a17e99ae64b878920d451cc814a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.445048 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa99492baf64897afd77f6466ae805c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.445092 27587 raft_consensus.cc:3037] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.445093 27659 leader_election.cc:302] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.445190 28774 raft_consensus.cc:2781] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.445223 28774 raft_consensus.cc:686] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.445258 28774 consensus_queue.cc:237] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.445454 28661 tablet_replica.cc:330] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.445544 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 32d965b7253c4f7582e32802e4ec73aa (DEFAULT_TABLE table=default.Table_a543dbd910a44984910275944aa9e670 [id=a8cd12a3c7364147b9ce763639338b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.445642 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d965b7253c4f7582e32802e4ec73aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.445704 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 771a298237a74dd0a1dc5097ca794b5d (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:15.445775 28661 raft_consensus.cc:2229] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.445827 28661 raft_consensus.cc:2258] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.445833 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 32d965b7253c4f7582e32802e4ec73aa (DEFAULT_TABLE table=default.Table_a543dbd910a44984910275944aa9e670 [id=a8cd12a3c7364147b9ce763639338b54]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.445904 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d965b7253c4f7582e32802e4ec73aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.445979 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 59299d27526640828eda00961363d0a6 with delete_type TABLET_DATA_DELETED (replaced by tablet f9f8911e604043c8a5b313ef62d206f0) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.446061 28664 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.446187 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 (DEFAULT_TABLE table=default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.446270 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c89b472abbe4fceb027714afa92a7b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.446619 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2e19130dfc744eea8079daaafd81a7b0" } new_table_name: "default.Table_2f1d3f2362414eeea600076e4ff8de6a" modify_external_catalogs: true
I20240503 02:09:15.448467 28661 ts_tablet_manager.cc:1875] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.448668 28664 tablet_bootstrap.cc:654] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.448685 27587 raft_consensus.cc:2454] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.448690 28653 tablet_replica.cc:330] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.448740 28653 raft_consensus.cc:2229] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.448771 28653 raft_consensus.cc:2258] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.448836 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c0b8f82c500549d29c0d1a41d8e111cf (table default.Table_72b3b65538eb417192d7f431d42c12cd [id=1661c2b9e1594f97a9cd619a8657107f]) successfully deleted
I20240503 02:09:15.448855 28653 ts_tablet_manager.cc:1875] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.448948 28654 ts_tablet_manager.cc:1875] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.449440 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 694e3a4f847e4f2d9d386cbb5c09842c (table default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240]). Expected version 1 got 0
I20240503 02:09:15.449502 27200 catalog_manager.cc:5484] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.449599 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.449792 27557 tablet_service.cc:1467] Processing CreateTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca (DEFAULT_TABLE table=default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.449813 27560 tablet_service.cc:1467] Processing CreateTablet for tablet c76ce5e466e64b5cae709aecaa1f4c0e (DEFAULT_TABLE table=default.Table_1a6378143d7f492196e138d6e4c73b59 [id=de2e32778b3b450f92aa92f114b42c44]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.449867 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48a9db1d1d4c129f07e4e0eb9918ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.449906 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76ce5e466e64b5cae709aecaa1f4c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.449906 27693 tablet_service.cc:1467] Processing CreateTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca (DEFAULT_TABLE table=default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.449941 27554 tablet_service.cc:1467] Processing CreateTablet for tablet f9f8911e604043c8a5b313ef62d206f0 (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.449971 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48a9db1d1d4c129f07e4e0eb9918ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.450016 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f8911e604043c8a5b313ef62d206f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.450510 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (DEFAULT_TABLE table=default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.450584 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4b1db49a3d4c1085b24c95509fe6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.450918 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (DEFAULT_TABLE table=default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.450996 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4b1db49a3d4c1085b24c95509fe6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.451169 27692 tablet_service.cc:1467] Processing CreateTablet for tablet c76ce5e466e64b5cae709aecaa1f4c0e (DEFAULT_TABLE table=default.Table_1a6378143d7f492196e138d6e4c73b59 [id=de2e32778b3b450f92aa92f114b42c44]), partition=RANGE (key) PARTITION UNBOUNDED
W20240503 02:09:15.451216 27198 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3900c1017d114230bc93d16ece225e3f consensus state has no leader
I20240503 02:09:15.451242 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76ce5e466e64b5cae709aecaa1f4c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.451740 27703 tablet_service.cc:1467] Processing CreateTablet for tablet f9f8911e604043c8a5b313ef62d206f0 (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.451807 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f8911e604043c8a5b313ef62d206f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.452628 28661 ts_tablet_manager.cc:1888] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.452674 28661 log.cc:1196] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/59299d27526640828eda00961363d0a6
I20240503 02:09:15.452840 28661 ts_tablet_manager.cc:1909] T 59299d27526640828eda00961363d0a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.453265 28655 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.453258 28698 tablet.cc:1722] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.453294 28656 tablet_bootstrap.cc:492] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.453311 28655 ts_tablet_manager.cc:1367] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.000s
I20240503 02:09:15.453370 28656 ts_tablet_manager.cc:1367] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240503 02:09:15.453495 28656 raft_consensus.cc:348] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.453495 28655 raft_consensus.cc:348] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.453545 28656 raft_consensus.cc:374] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.453545 28655 raft_consensus.cc:374] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.453565 28656 raft_consensus.cc:729] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.453565 28655 raft_consensus.cc:729] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.453596 28656 consensus_queue.cc:260] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.453604 28655 consensus_queue.cc:260] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.453698 28655 ts_tablet_manager.cc:1398] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.453706 28656 ts_tablet_manager.cc:1398] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.453750 28656 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.453752 28655 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.454077 28655 tablet_bootstrap.cc:654] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.454077 28656 tablet_bootstrap.cc:654] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.453605 28653 ts_tablet_manager.cc:1888] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.454692 28653 log.cc:1196] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/388e82752def4a31b3d8bec18a478273
I20240503 02:09:15.454871 28653 ts_tablet_manager.cc:1909] T 388e82752def4a31b3d8bec18a478273 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.454866 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f011cd6264814548a989da59664aaf6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.454949 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f011cd6264814548a989da59664aaf6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.455052 28653 tablet_replica.cc:330] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.455111 28653 raft_consensus.cc:2229] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.455138 28653 raft_consensus.cc:2258] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.455420 28686 tablet.cc:1722] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.455638 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet dc51af2f65424fb0a7d26e13e4cd72ce from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.455991 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 388e82752def4a31b3d8bec18a478273 (table default.Table_bf60bed921ad4219bd0b452cf484c521 [id=38784dc3ff464c6a8152e37f4d330fd6]) successfully deleted
I20240503 02:09:15.456107 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7310e9433672464690114ff02b5c773f" } new_table_name: "default.Table_2c38711e62194693a4f5e082589671f1" modify_external_catalogs: true
I20240503 02:09:15.456298 28653 ts_tablet_manager.cc:1875] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.456349 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca (DEFAULT_TABLE table=default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.456410 27419 tablet_service.cc:1467] Processing CreateTablet for tablet f9f8911e604043c8a5b313ef62d206f0 (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.456430 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec48a9db1d1d4c129f07e4e0eb9918ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.456485 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9f8911e604043c8a5b313ef62d206f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.457340 28653 ts_tablet_manager.cc:1888] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.457351 27722 raft_consensus.cc:1261] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.457391 28653 log.cc:1196] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/771a298237a74dd0a1dc5097ca794b5d
I20240503 02:09:15.457495 27433 tablet_service.cc:1467] Processing CreateTablet for tablet c76ce5e466e64b5cae709aecaa1f4c0e (DEFAULT_TABLE table=default.Table_1a6378143d7f492196e138d6e4c73b59 [id=de2e32778b3b450f92aa92f114b42c44]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.457530 28653 ts_tablet_manager.cc:1909] T 771a298237a74dd0a1dc5097ca794b5d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.457540 27848 consensus_queue.cc:1035] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.457577 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76ce5e466e64b5cae709aecaa1f4c0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.457648 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (DEFAULT_TABLE table=default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.457718 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4b1db49a3d4c1085b24c95509fe6a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.458422 27451 raft_consensus.cc:1261] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.459147 28653 tablet_replica.cc:330] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.459162 27848 consensus_queue.cc:1035] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.459264 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 59299d27526640828eda00961363d0a6 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:15.459478 28656 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.459517 28656 ts_tablet_manager.cc:1367] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:15.459640 28656 raft_consensus.cc:348] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.459686 28656 raft_consensus.cc:374] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.459702 28656 raft_consensus.cc:729] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.459728 28656 consensus_queue.cc:260] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.459803 28656 ts_tablet_manager.cc:1398] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.459821 28664 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.459859 28656 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.459861 28664 ts_tablet_manager.cc:1367] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240503 02:09:15.459985 28664 raft_consensus.cc:348] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.460050 28664 raft_consensus.cc:374] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.460067 28664 raft_consensus.cc:729] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.460104 28664 consensus_queue.cc:260] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.460176 28656 tablet_bootstrap.cc:654] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.460171 28653 raft_consensus.cc:2229] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.460181 28664 ts_tablet_manager.cc:1398] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.460219 28653 raft_consensus.cc:2258] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.460238 28664 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.460425 28653 ts_tablet_manager.cc:1875] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.460489 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet dc51af2f65424fb0a7d26e13e4cd72ce
I20240503 02:09:15.460554 27198 catalog_manager.cc:6554] ReplaceTablet: tablet dc51af2f65424fb0a7d26e13e4cd72ce deleted and replaced by tablet b0d7641f83dc4d029ecd170a2d14888b
I20240503 02:09:15.460579 28664 tablet_bootstrap.cc:654] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.460592 28698 tablet.cc:1722] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.460801 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet c583b44cecad4e93a50fddae3a0035b7 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.461086 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce with delete_type TABLET_DATA_DELETED (replaced by tablet b0d7641f83dc4d029ecd170a2d14888b) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.461180 28661 tablet_replica.cc:330] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.461213 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce with delete_type TABLET_DATA_DELETED (replaced by tablet b0d7641f83dc4d029ecd170a2d14888b) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.461249 28661 raft_consensus.cc:2229] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.461302 28694 tablet.cc:1722] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.461192 28706 tablet.cc:1722] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.461302 28661 raft_consensus.cc:2258] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.461709 28661 ts_tablet_manager.cc:1875] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.461735 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 771a298237a74dd0a1dc5097ca794b5d (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:15.461746 28656 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.461777 28656 ts_tablet_manager.cc:1367] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.461890 28656 raft_consensus.cc:348] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.461941 28656 raft_consensus.cc:374] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.461957 28656 raft_consensus.cc:729] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.461992 28656 consensus_queue.cc:260] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.462066 28656 ts_tablet_manager.cc:1398] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.462109 28656 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.462255 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet dc51af2f65424fb0a7d26e13e4cd72ce with delete_type TABLET_DATA_DELETED (replaced by tablet b0d7641f83dc4d029ecd170a2d14888b) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.462426 28656 tablet_bootstrap.cc:654] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.462850 28661 ts_tablet_manager.cc:1888] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.462916 28661 log.cc:1196] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/dc51af2f65424fb0a7d26e13e4cd72ce
I20240503 02:09:15.463093 28661 ts_tablet_manager.cc:1909] T dc51af2f65424fb0a7d26e13e4cd72ce P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.463693 28664 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.463737 28664 ts_tablet_manager.cc:1367] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.463774 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_aded67e380fd48b987de136312959fd5" } modify_external_catalogs: true
I20240503 02:09:15.463825 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_aded67e380fd48b987de136312959fd5" } modify_external_catalogs: true
I20240503 02:09:15.463848 28664 raft_consensus.cc:348] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.463891 28664 raft_consensus.cc:374] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.463907 28664 raft_consensus.cc:729] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.463940 28664 consensus_queue.cc:260] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.464038 28664 ts_tablet_manager.cc:1398] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.464085 28664 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.464377 28664 tablet_bootstrap.cc:654] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.465519 27449 raft_consensus.cc:1261] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.465780 28656 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.465831 28656 ts_tablet_manager.cc:1367] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.465942 28656 raft_consensus.cc:348] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.465998 28656 raft_consensus.cc:374] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.465989 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet dc51af2f65424fb0a7d26e13e4cd72ce (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:15.466017 28656 raft_consensus.cc:729] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.466061 28656 consensus_queue.cc:260] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.466140 28656 ts_tablet_manager.cc:1398] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.466197 28656 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.466396 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c583b44cecad4e93a50fddae3a0035b7
I20240503 02:09:15.466460 27198 catalog_manager.cc:6554] ReplaceTablet: tablet c583b44cecad4e93a50fddae3a0035b7 deleted and replaced by tablet d5fb40e254ab40f09cc72093bf50c91b
I20240503 02:09:15.466480 27703 tablet_service.cc:1467] Processing CreateTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 (DEFAULT_TABLE table=default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.466529 28656 tablet_bootstrap.cc:654] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.466558 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6244d9ac8b40aab5f6e2a40cda8463. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.466681 28653 ts_tablet_manager.cc:1888] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.466719 28653 log.cc:1196] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/59299d27526640828eda00961363d0a6
I20240503 02:09:15.466778 28774 consensus_queue.cc:1035] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.466900 28653 ts_tablet_manager.cc:1909] T 59299d27526640828eda00961363d0a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.467196 27587 raft_consensus.cc:1261] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.467228 28656 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.467262 28656 ts_tablet_manager.cc:1367] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.467439 28653 tablet_replica.cc:330] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.467456 28656 raft_consensus.cc:348] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.467511 28656 raft_consensus.cc:374] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.467548 28656 raft_consensus.cc:729] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.467574 28656 consensus_queue.cc:260] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.467581 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet c583b44cecad4e93a50fddae3a0035b7 with delete_type TABLET_DATA_DELETED (replaced by tablet d5fb40e254ab40f09cc72093bf50c91b) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.467576 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d400c7ed3eac45339ebf652072ddb986" } modify_external_catalogs: true
I20240503 02:09:15.467628 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d400c7ed3eac45339ebf652072ddb986" } modify_external_catalogs: true
I20240503 02:09:15.467658 28661 tablet_replica.cc:330] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.467669 28656 ts_tablet_manager.cc:1398] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.467708 28661 raft_consensus.cc:2229] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.467749 28661 pending_rounds.cc:62] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.467772 28661 pending_rounds.cc:68] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027302793216 op_type: NO_OP noop_request { }
I20240503 02:09:15.467815 28661 raft_consensus.cc:2866] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.467834 28661 raft_consensus.cc:2258] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.467633 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_eec82c477df84e9dbd2b9874505d9e48" } new_table_name: "default.Table_8f3497b3f2fd46abb20e2efdefdefb34" modify_external_catalogs: true
I20240503 02:09:15.467908 28774 consensus_queue.cc:1035] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.468295 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet c583b44cecad4e93a50fddae3a0035b7 with delete_type TABLET_DATA_DELETED (replaced by tablet d5fb40e254ab40f09cc72093bf50c91b) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.468456 27557 tablet_service.cc:1467] Processing CreateTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 (DEFAULT_TABLE table=default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.468479 28661 ts_tablet_manager.cc:1875] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.468729 28656 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.469074 28656 tablet_bootstrap.cc:654] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.469084 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6244d9ac8b40aab5f6e2a40cda8463. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.469630 28774 raft_consensus.cc:482] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.469679 28774 raft_consensus.cc:504] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.469779 28774 leader_election.cc:288] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.470008 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d965b7253c4f7582e32802e4ec73aa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.470094 28661 ts_tablet_manager.cc:1888] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.470162 28661 log.cc:1196] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c583b44cecad4e93a50fddae3a0035b7
W20240503 02:09:15.470192 27659 leader_election.cc:341] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.470340 28661 ts_tablet_manager.cc:1909] T c583b44cecad4e93a50fddae3a0035b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.470458 28653 raft_consensus.cc:2229] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.470535 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 3858a2107d6f434aaad8156a4f617115 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.470571 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d965b7253c4f7582e32802e4ec73aa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.470633 27582 raft_consensus.cc:2454] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.470780 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c583b44cecad4e93a50fddae3a0035b7 (table default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]) successfully deleted
I20240503 02:09:15.470779 27656 leader_election.cc:302] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:15.470799 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 59299d27526640828eda00961363d0a6 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:15.470863 28774 raft_consensus.cc:2781] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.470904 28774 raft_consensus.cc:482] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.470922 28774 raft_consensus.cc:3037] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.470985 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e3c9273bebae49509f713f02bc050cf5
I20240503 02:09:15.471515 28774 raft_consensus.cc:504] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.471611 28774 leader_election.cc:288] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.472010 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c9273bebae49509f713f02bc050cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.472184 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c9273bebae49509f713f02bc050cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.472506 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 694e3a4f847e4f2d9d386cbb5c09842c (table default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240]). Expected version 1 got 0
I20240503 02:09:15.472589 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 11b157b6f5d04c31ac019de2d800d112 (table default.Table_2f1d3f2362414eeea600076e4ff8de6a [id=9205c1ddc30a4442aa89c0b58b5b7700]). Expected version 1 got 0
I20240503 02:09:15.472621 28661 tablet_replica.cc:330] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.472618 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d965b7253c4f7582e32802e4ec73aa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.472666 27582 raft_consensus.cc:3037] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.472671 28661 raft_consensus.cc:2229] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.472698 28661 raft_consensus.cc:2258] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.470840 28710 tablet.cc:1722] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.472780 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9dffa71e4ead479b801922f8a385be4e
I20240503 02:09:15.471169 27427 tablet_service.cc:1467] Processing CreateTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 (DEFAULT_TABLE table=default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.472864 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6244d9ac8b40aab5f6e2a40cda8463. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.473256 27582 raft_consensus.cc:2454] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.473416 27693 tablet_service.cc:1467] Processing CreateTablet for tablet b0d7641f83dc4d029ecd170a2d14888b (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.473501 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d7641f83dc4d029ecd170a2d14888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.473657 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffa71e4ead479b801922f8a385be4e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.471184 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet c583b44cecad4e93a50fddae3a0035b7 with delete_type TABLET_DATA_DELETED (replaced by tablet d5fb40e254ab40f09cc72093bf50c91b) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.471668 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c9273bebae49509f713f02bc050cf5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.472506 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d965b7253c4f7582e32802e4ec73aa" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
W20240503 02:09:15.473985 27659 leader_election.cc:341] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.474215 28661 ts_tablet_manager.cc:1875] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.474509 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffa71e4ead479b801922f8a385be4e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.474264 27656 leader_election.cc:302] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:15.474514 27433 tablet_service.cc:1467] Processing CreateTablet for tablet b0d7641f83dc4d029ecd170a2d14888b (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.474792 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d7641f83dc4d029ecd170a2d14888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.474984 28774 raft_consensus.cc:2781] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.475028 28774 raft_consensus.cc:686] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.475059 28774 consensus_queue.cc:237] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.475086 27722 raft_consensus.cc:1261] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.475189 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3858a2107d6f434aaad8156a4f617115
I20240503 02:09:15.475255 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 3858a2107d6f434aaad8156a4f617115 deleted and replaced by tablet 59d556e0a8904b519fc0656e4692d4ca
I20240503 02:09:15.475242 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffa71e4ead479b801922f8a385be4e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.475303 28410 consensus_queue.cc:1035] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.475385 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 3858a2107d6f434aaad8156a4f617115 with delete_type TABLET_DATA_DELETED (replaced by tablet 59d556e0a8904b519fc0656e4692d4ca) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.475920 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 3858a2107d6f434aaad8156a4f617115 with delete_type TABLET_DATA_DELETED (replaced by tablet 59d556e0a8904b519fc0656e4692d4ca) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.476290 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 3858a2107d6f434aaad8156a4f617115 with delete_type TABLET_DATA_DELETED (replaced by tablet 59d556e0a8904b519fc0656e4692d4ca) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.476338 27451 raft_consensus.cc:1261] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.476379 27557 tablet_service.cc:1467] Processing CreateTablet for tablet b0d7641f83dc4d029ecd170a2d14888b (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.476460 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d7641f83dc4d029ecd170a2d14888b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.476869 28661 ts_tablet_manager.cc:1888] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.476929 28661 log.cc:1196] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e3c9273bebae49509f713f02bc050cf5
I20240503 02:09:15.477073 28661 ts_tablet_manager.cc:1909] T e3c9273bebae49509f713f02bc050cf5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.477205 28653 raft_consensus.cc:2258] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.477566 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e3c9273bebae49509f713f02bc050cf5 (table default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]) successfully deleted
I20240503 02:09:15.477545 28661 tablet_replica.cc:330] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.477838 27693 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb40e254ab40f09cc72093bf50c91b (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.477896 28661 raft_consensus.cc:2229] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.477995 28661 raft_consensus.cc:2258] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.478125 28653 ts_tablet_manager.cc:1875] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.478399 27557 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb40e254ab40f09cc72093bf50c91b (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.478489 27433 tablet_service.cc:1467] Processing CreateTablet for tablet d5fb40e254ab40f09cc72093bf50c91b (DEFAULT_TABLE table=default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.478534 28410 consensus_queue.cc:1035] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.478595 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb40e254ab40f09cc72093bf50c91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.478813 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb40e254ab40f09cc72093bf50c91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.479383 28661 ts_tablet_manager.cc:1875] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.478004 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fb40e254ab40f09cc72093bf50c91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.480854 28654 ts_tablet_manager.cc:1888] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.480898 28654 log.cc:1196] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/59299d27526640828eda00961363d0a6
I20240503 02:09:15.481055 28654 ts_tablet_manager.cc:1909] T 59299d27526640828eda00961363d0a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.481098 28661 ts_tablet_manager.cc:1888] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.481098 28653 ts_tablet_manager.cc:1888] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.481153 28661 log.cc:1196] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9dffa71e4ead479b801922f8a385be4e
I20240503 02:09:15.481153 28653 log.cc:1196] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/dc51af2f65424fb0a7d26e13e4cd72ce
I20240503 02:09:15.481563 28661 ts_tablet_manager.cc:1909] T 9dffa71e4ead479b801922f8a385be4e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.481590 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240] alter complete (version 1)
I20240503 02:09:15.481834 28661 tablet_replica.cc:330] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.481860 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9dffa71e4ead479b801922f8a385be4e (table default.Table_d400c7ed3eac45339ebf652072ddb986 [id=3fc56404e8894e06afad9ccc8d82cf89]) successfully deleted
I20240503 02:09:15.481890 28661 raft_consensus.cc:2229] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.481923 28661 pending_rounds.cc:62] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 2 pending ops.
I20240503 02:09:15.481938 28661 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7023420028785262592 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3858a2107d6f434aaad8156a4f617115" schema { columns { id: 0 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_2c38711e62194693a4f5e082589671f1" dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" new_extra_config { } }
W20240503 02:09:15.482007 28661 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43176: Aborted: Op aborted
I20240503 02:09:15.482127 28661 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027741093888 op_type: NO_OP noop_request { }
I20240503 02:09:15.482174 28661 raft_consensus.cc:2866] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.481839 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 59d556e0a8904b519fc0656e4692d4ca (DEFAULT_TABLE table=default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.482220 28661 raft_consensus.cc:2258] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:15.482232 27189 catalog_manager.cc:4909] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): alter failed for tablet 3858a2107d6f434aaad8156a4f617115 (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]): Aborted: Op aborted
I20240503 02:09:15.482299 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d556e0a8904b519fc0656e4692d4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.482410 28661 ts_tablet_manager.cc:1875] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.482857 28653 ts_tablet_manager.cc:1909] T dc51af2f65424fb0a7d26e13e4cd72ce P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.483192 28653 tablet_replica.cc:330] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.483244 28653 raft_consensus.cc:2229] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.483270 28653 raft_consensus.cc:2258] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:15.483304 27197 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:15.483347 28653 ts_tablet_manager.cc:1875] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.483484 28661 ts_tablet_manager.cc:1888] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.483522 28661 log.cc:1196] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3858a2107d6f434aaad8156a4f617115
I20240503 02:09:15.483587 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_2f1d3f2362414eeea600076e4ff8de6a [id=9205c1ddc30a4442aa89c0b58b5b7700] alter complete (version 1)
I20240503 02:09:15.483690 28661 ts_tablet_manager.cc:1909] T 3858a2107d6f434aaad8156a4f617115 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.484246 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.484575 27197 catalog_manager.cc:5484] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.484637 28653 ts_tablet_manager.cc:1888] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.484678 28653 log.cc:1196] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c583b44cecad4e93a50fddae3a0035b7
I20240503 02:09:15.484683 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.484853 28653 ts_tablet_manager.cc:1909] T c583b44cecad4e93a50fddae3a0035b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.485158 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3858a2107d6f434aaad8156a4f617115 (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:15.485199 28653 tablet_replica.cc:330] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.485251 28653 raft_consensus.cc:2229] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.485275 28653 raft_consensus.cc:2258] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.485349 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet dc51af2f65424fb0a7d26e13e4cd72ce (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:15.485407 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c583b44cecad4e93a50fddae3a0035b7 (table default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]) successfully deleted
I20240503 02:09:15.485524 28653 ts_tablet_manager.cc:1875] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.485817 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 59d556e0a8904b519fc0656e4692d4ca (DEFAULT_TABLE table=default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.485898 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d556e0a8904b519fc0656e4692d4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.486269 27723 raft_consensus.cc:1261] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.486464 27848 consensus_queue.cc:1035] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.486763 28654 tablet_replica.cc:330] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.486784 27453 raft_consensus.cc:1261] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.486820 28654 raft_consensus.cc:2229] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.486850 28654 raft_consensus.cc:2258] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.487017 28653 ts_tablet_manager.cc:1888] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.487030 28654 ts_tablet_manager.cc:1875] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.487057 28653 log.cc:1196] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e3c9273bebae49509f713f02bc050cf5
I20240503 02:09:15.487201 28653 ts_tablet_manager.cc:1909] T e3c9273bebae49509f713f02bc050cf5 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.487484 28653 tablet_replica.cc:330] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.487596 28653 raft_consensus.cc:2229] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.487673 28653 raft_consensus.cc:2258] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.487726 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.487968 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 59299d27526640828eda00961363d0a6 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:15.488095 28656 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.488135 28656 ts_tablet_manager.cc:1367] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240503 02:09:15.488282 28656 raft_consensus.cc:348] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.488333 28656 raft_consensus.cc:374] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.488353 28656 raft_consensus.cc:729] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.488396 28656 consensus_queue.cc:260] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.488483 28656 ts_tablet_manager.cc:1398] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.488529 28656 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.488530 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.488898 28656 tablet_bootstrap.cc:654] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.489197 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
W20240503 02:09:15.489269 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet f9f8911e604043c8a5b313ef62d206f0 consensus state has no leader
I20240503 02:09:15.489462 28653 ts_tablet_manager.cc:1875] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.489516 28656 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.489553 28656 ts_tablet_manager.cc:1367] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.489667 28656 raft_consensus.cc:348] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.489712 28656 raft_consensus.cc:374] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.489728 28656 raft_consensus.cc:729] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.489760 28656 consensus_queue.cc:260] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.489846 28656 ts_tablet_manager.cc:1398] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.489917 28656 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.490229 28656 tablet_bootstrap.cc:654] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.490368 28654 ts_tablet_manager.cc:1888] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.490411 28654 log.cc:1196] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/dc51af2f65424fb0a7d26e13e4cd72ce
I20240503 02:09:15.490726 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.491091 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e3c9273bebae49509f713f02bc050cf5 (table default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]) successfully deleted
I20240503 02:09:15.491256 28507 consensus_queue.cc:1035] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.491432 28698 tablet.cc:1722] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.491461 28653 ts_tablet_manager.cc:1888] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.491494 28653 log.cc:1196] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9dffa71e4ead479b801922f8a385be4e
I20240503 02:09:15.491650 28653 ts_tablet_manager.cc:1909] T 9dffa71e4ead479b801922f8a385be4e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.492019 28653 tablet_replica.cc:330] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.492084 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9dffa71e4ead479b801922f8a385be4e (table default.Table_d400c7ed3eac45339ebf652072ddb986 [id=3fc56404e8894e06afad9ccc8d82cf89]) successfully deleted
I20240503 02:09:15.492250 28653 raft_consensus.cc:2229] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.492390 28653 pending_rounds.cc:62] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 2 pending ops.
I20240503 02:09:15.492614 28686 tablet.cc:1722] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.492892 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89] alter complete (version 1)
I20240503 02:09:15.492878 28653 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7023420028785262592 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3858a2107d6f434aaad8156a4f617115" schema { columns { id: 0 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_2c38711e62194693a4f5e082589671f1" dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" new_extra_config { } }
I20240503 02:09:15.492956 27848 raft_consensus.cc:482] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.492992 27848 raft_consensus.cc:504] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.493089 27848 leader_election.cc:288] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.493093 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.493129 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.493077 28653 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027741093888 op_type: NO_OP noop_request { }
I20240503 02:09:15.493211 28653 raft_consensus.cc:2866] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.493199 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.493234 28653 raft_consensus.cc:2258] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:15.493551 27523 leader_election.cc:341] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.493541 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:15.494122 27524 leader_election.cc:341] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.494164 27524 leader_election.cc:302] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:15.494223 28410 raft_consensus.cc:2726] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:15.494383 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]). Expected version 1 got 0
I20240503 02:09:15.495173 28698 tablet.cc:1722] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.495201 28654 ts_tablet_manager.cc:1909] T dc51af2f65424fb0a7d26e13e4cd72ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.495358 28706 tablet.cc:1722] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.495448 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet dc51af2f65424fb0a7d26e13e4cd72ce (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:15.495834 28654 tablet_replica.cc:330] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.495913 28654 raft_consensus.cc:2229] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.495983 28654 raft_consensus.cc:2258] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.496146 28654 ts_tablet_manager.cc:1875] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.496145 28686 tablet.cc:1722] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.497896 28654 ts_tablet_manager.cc:1888] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.497941 28654 log.cc:1196] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c583b44cecad4e93a50fddae3a0035b7
I20240503 02:09:15.498080 28654 ts_tablet_manager.cc:1909] T c583b44cecad4e93a50fddae3a0035b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.498201 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 59d556e0a8904b519fc0656e4692d4ca (DEFAULT_TABLE table=default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.498266 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59d556e0a8904b519fc0656e4692d4ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.498437 28654 tablet_replica.cc:330] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.498514 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c583b44cecad4e93a50fddae3a0035b7 (table default.Table_ef7ad145af794a2fab93caf50bce1418 [id=97264f1a599a4a9680a9f284e2633241]) successfully deleted
I20240503 02:09:15.498535 28654 raft_consensus.cc:2229] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.498584 28654 pending_rounds.cc:62] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:15.498611 28654 pending_rounds.cc:68] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027750330368 op_type: NO_OP noop_request { }
I20240503 02:09:15.498641 28654 raft_consensus.cc:2866] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.498657 28654 raft_consensus.cc:2258] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.498821 28654 ts_tablet_manager.cc:1875] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.499439 28655 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.499486 28655 ts_tablet_manager.cc:1367] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.001s
I20240503 02:09:15.499660 28655 raft_consensus.cc:348] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.499723 28655 raft_consensus.cc:374] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.499739 28655 raft_consensus.cc:729] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.499775 28655 consensus_queue.cc:260] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.499855 28655 ts_tablet_manager.cc:1398] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.499887 28654 ts_tablet_manager.cc:1888] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.499930 28654 log.cc:1196] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e3c9273bebae49509f713f02bc050cf5
I20240503 02:09:15.499994 28655 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.500319 28655 tablet_bootstrap.cc:654] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.500479 28654 ts_tablet_manager.cc:1909] T e3c9273bebae49509f713f02bc050cf5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
W20240503 02:09:15.499475 29081 consensus_queue.cc:1159] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bd4b79bd8b57443faa6fa5e464570188" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20240503 02:09:15.500917 28654 tablet_replica.cc:330] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.501241 28654 raft_consensus.cc:2229] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.501281 28654 raft_consensus.cc:2258] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.501262 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e3c9273bebae49509f713f02bc050cf5 (table default.Table_aded67e380fd48b987de136312959fd5 [id=202f8b78e8d141fda5279c7683d56b96]) successfully deleted
I20240503 02:09:15.501554 28654 ts_tablet_manager.cc:1875] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.501729 28653 ts_tablet_manager.cc:1875] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.502630 28654 ts_tablet_manager.cc:1888] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.502673 28654 log.cc:1196] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9dffa71e4ead479b801922f8a385be4e
I20240503 02:09:15.502825 28654 ts_tablet_manager.cc:1909] T 9dffa71e4ead479b801922f8a385be4e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.502884 28664 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.502933 28664 ts_tablet_manager.cc:1367] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.000s
I20240503 02:09:15.503049 28656 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.503074 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9dffa71e4ead479b801922f8a385be4e (table default.Table_d400c7ed3eac45339ebf652072ddb986 [id=3fc56404e8894e06afad9ccc8d82cf89]) successfully deleted
I20240503 02:09:15.503085 28656 ts_tablet_manager.cc:1367] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:15.503059 28664 raft_consensus.cc:348] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.503111 28664 raft_consensus.cc:374] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.503129 28664 raft_consensus.cc:729] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.503140 28654 tablet_replica.cc:330] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.503162 28664 consensus_queue.cc:260] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.503245 28664 ts_tablet_manager.cc:1398] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.503326 28664 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.503330 28656 raft_consensus.cc:348] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.503386 28656 raft_consensus.cc:374] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.503407 28656 raft_consensus.cc:729] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.503444 28656 consensus_queue.cc:260] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.503571 28656 ts_tablet_manager.cc:1398] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:15.503618 28656 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.503667 28664 tablet_bootstrap.cc:654] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.503937 28656 tablet_bootstrap.cc:654] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.504252 28653 ts_tablet_manager.cc:1888] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.504298 28653 log.cc:1196] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3858a2107d6f434aaad8156a4f617115
I20240503 02:09:15.504397 28673 mvcc.cc:205] Tried to move back new op lower bound from 7023420028823769088 to 7023420027739000832. Current Snapshot: MvccSnapshot[applied={T|T < 7023420028823769088}]
I20240503 02:09:15.503429 28654 raft_consensus.cc:2229] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.504491 28654 pending_rounds.cc:62] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 2 pending ops.
I20240503 02:09:15.504515 28654 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7023420028785262592 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3858a2107d6f434aaad8156a4f617115" schema { columns { id: 0 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_2c38711e62194693a4f5e082589671f1" dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" new_extra_config { } }
I20240503 02:09:15.504602 28654 pending_rounds.cc:68] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027741093888 op_type: NO_OP noop_request { }
I20240503 02:09:15.504628 28654 raft_consensus.cc:2866] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.504647 28654 raft_consensus.cc:2258] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.504773 28706 tablet.cc:1722] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.504946 28654 ts_tablet_manager.cc:1875] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.504971 28656 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.505000 28656 ts_tablet_manager.cc:1367] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.505110 28656 raft_consensus.cc:348] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.505175 28656 raft_consensus.cc:374] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.505173 28653 ts_tablet_manager.cc:1909] T 3858a2107d6f434aaad8156a4f617115 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.505194 28656 raft_consensus.cc:729] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.505227 28656 consensus_queue.cc:260] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.505300 28656 ts_tablet_manager.cc:1398] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.505368 28656 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.505702 28656 tablet_bootstrap.cc:654] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.506146 28656 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.506196 28656 ts_tablet_manager.cc:1367] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.506321 28656 raft_consensus.cc:348] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.506382 28656 raft_consensus.cc:374] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.506400 28656 raft_consensus.cc:729] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.506434 28656 consensus_queue.cc:260] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.506526 28656 ts_tablet_manager.cc:1398] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.506579 28656 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.506901 28656 tablet_bootstrap.cc:654] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.507346 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3858a2107d6f434aaad8156a4f617115 (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:15.507535 28655 tablet_bootstrap.cc:492] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.507580 28655 ts_tablet_manager.cc:1367] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:15.507690 28655 raft_consensus.cc:348] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.507757 28655 raft_consensus.cc:374] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.507773 28655 raft_consensus.cc:729] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.507807 28655 consensus_queue.cc:260] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.507892 28655 ts_tablet_manager.cc:1398] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.507937 28655 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.508230 28655 tablet_bootstrap.cc:654] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.508818 28655 tablet_bootstrap.cc:492] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.508848 28654 ts_tablet_manager.cc:1888] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.508864 28655 ts_tablet_manager.cc:1367] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.508893 28654 log.cc:1196] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3858a2107d6f434aaad8156a4f617115
I20240503 02:09:15.508989 28655 raft_consensus.cc:348] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.509037 28655 raft_consensus.cc:374] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.509055 28655 raft_consensus.cc:729] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.509066 28654 ts_tablet_manager.cc:1909] T 3858a2107d6f434aaad8156a4f617115 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.509092 28655 consensus_queue.cc:260] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.509177 28655 ts_tablet_manager.cc:1398] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.509254 28655 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.509459 28664 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.509506 28664 ts_tablet_manager.cc:1367] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:15.509627 28655 tablet_bootstrap.cc:654] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.509626 28664 raft_consensus.cc:348] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.509647 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3858a2107d6f434aaad8156a4f617115 (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]) successfully deleted
I20240503 02:09:15.509678 28664 raft_consensus.cc:374] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.509699 28664 raft_consensus.cc:729] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.509737 28664 consensus_queue.cc:260] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.509828 28664 ts_tablet_manager.cc:1398] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.509896 28664 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.510221 28664 tablet_bootstrap.cc:654] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.511034 28664 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.511076 28664 ts_tablet_manager.cc:1367] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.511265 28664 raft_consensus.cc:348] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.511323 28664 raft_consensus.cc:374] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.511340 28664 raft_consensus.cc:729] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.511377 28664 consensus_queue.cc:260] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.511466 28664 ts_tablet_manager.cc:1398] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.511868 28664 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.512224 28664 tablet_bootstrap.cc:654] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:15.512730 27189 catalog_manager.cc:4903] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): alter failed for tablet 3858a2107d6f434aaad8156a4f617115 (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]),no further retry: Not found: Tablet not found: 3858a2107d6f434aaad8156a4f617115
W20240503 02:09:15.512867 27187 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet f9f8911e604043c8a5b313ef62d206f0 consensus state has no leader
I20240503 02:09:15.513067 28656 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.513070 28664 tablet_bootstrap.cc:492] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.513109 28656 ts_tablet_manager.cc:1367] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:15.513110 28664 ts_tablet_manager.cc:1367] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.513214 28664 raft_consensus.cc:348] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.513222 28656 raft_consensus.cc:348] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.513257 28664 raft_consensus.cc:374] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.513270 28664 raft_consensus.cc:729] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.513268 28656 raft_consensus.cc:374] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.513288 28656 raft_consensus.cc:729] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.513346 28656 consensus_queue.cc:260] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.513311 28664 consensus_queue.cc:260] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.513433 28664 ts_tablet_manager.cc:1398] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.513485 28664 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.513486 28656 ts_tablet_manager.cc:1398] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.513546 28656 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.513830 28664 tablet_bootstrap.cc:654] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.513924 28656 tablet_bootstrap.cc:654] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.514501 28664 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.514546 28664 ts_tablet_manager.cc:1367] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.514657 28664 raft_consensus.cc:348] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.514683 28655 tablet_bootstrap.cc:492] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.514704 28664 raft_consensus.cc:374] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.514724 28664 raft_consensus.cc:729] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.514724 28655 ts_tablet_manager.cc:1367] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:15.514732 28656 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.514765 28656 ts_tablet_manager.cc:1367] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.514757 28664 consensus_queue.cc:260] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.514847 28664 ts_tablet_manager.cc:1398] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.514842 28655 raft_consensus.cc:348] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.514891 28655 raft_consensus.cc:374] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.514897 28664 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.514912 28655 raft_consensus.cc:729] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.514885 28656 raft_consensus.cc:348] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.514931 28656 raft_consensus.cc:374] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.514950 28656 raft_consensus.cc:729] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.514947 28655 consensus_queue.cc:260] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.514994 28656 consensus_queue.cc:260] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.515025 28655 ts_tablet_manager.cc:1398] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.515074 28656 ts_tablet_manager.cc:1398] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.515085 28655 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.515116 28656 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.515208 28664 tablet_bootstrap.cc:654] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.515430 28656 tablet_bootstrap.cc:654] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.516198 28664 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.516235 28664 ts_tablet_manager.cc:1367] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.516269 28655 tablet_bootstrap.cc:654] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.516666 28664 raft_consensus.cc:348] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.516723 28664 raft_consensus.cc:374] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.516741 28664 raft_consensus.cc:729] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.516788 28664 consensus_queue.cc:260] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.517192 28664 ts_tablet_manager.cc:1398] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.517230 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.517251 28664 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.517925 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.517940 28664 tablet_bootstrap.cc:654] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.518482 28655 tablet_bootstrap.cc:492] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.518524 28655 ts_tablet_manager.cc:1367] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:15.518591 28656 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.518633 28656 ts_tablet_manager.cc:1367] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.518661 28655 raft_consensus.cc:348] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.518718 28655 raft_consensus.cc:374] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.518733 28655 raft_consensus.cc:729] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.518721 28656 raft_consensus.cc:348] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.518760 28656 raft_consensus.cc:374] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.518777 28656 raft_consensus.cc:729] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.518786 28655 consensus_queue.cc:260] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.518827 28656 consensus_queue.cc:260] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.518924 28656 ts_tablet_manager.cc:1398] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.519495 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.519579 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
W20240503 02:09:15.519663 27194 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 59d556e0a8904b519fc0656e4692d4ca consensus state has no leader
I20240503 02:09:15.519801 28655 ts_tablet_manager.cc:1398] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.520267 28655 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.520519 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.520783 28655 tablet_bootstrap.cc:654] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.521255 28655 tablet_bootstrap.cc:492] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.521255 28664 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.521294 28664 ts_tablet_manager.cc:1367] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.521294 28655 ts_tablet_manager.cc:1367] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.521467 28664 raft_consensus.cc:348] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.521521 28664 raft_consensus.cc:374] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.521513 28655 raft_consensus.cc:348] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.521539 28664 raft_consensus.cc:729] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.521553 28655 raft_consensus.cc:374] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.521572 28655 raft_consensus.cc:729] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.521577 28664 consensus_queue.cc:260] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.521605 28655 consensus_queue.cc:260] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.521660 28664 ts_tablet_manager.cc:1398] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.521697 28655 ts_tablet_manager.cc:1398] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.521713 28664 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.521752 28655 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.522045 28664 tablet_bootstrap.cc:654] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.522084 28655 tablet_bootstrap.cc:654] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.522843 28655 tablet_bootstrap.cc:492] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.522893 28655 ts_tablet_manager.cc:1367] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.523003 28655 raft_consensus.cc:348] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.523059 28655 raft_consensus.cc:374] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.523075 28655 raft_consensus.cc:729] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.523118 28655 consensus_queue.cc:260] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.523197 28655 ts_tablet_manager.cc:1398] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.523305 28655 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.523629 28655 tablet_bootstrap.cc:654] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.524148 28655 tablet_bootstrap.cc:492] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.524194 28655 ts_tablet_manager.cc:1367] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.524303 28655 raft_consensus.cc:348] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.524346 28655 raft_consensus.cc:374] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.524360 28655 raft_consensus.cc:729] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.524394 28655 consensus_queue.cc:260] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.524685 28655 ts_tablet_manager.cc:1398] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.524749 28655 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.525225 28655 tablet_bootstrap.cc:654] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.525808 28655 tablet_bootstrap.cc:492] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.525853 28655 ts_tablet_manager.cc:1367] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.525969 28655 raft_consensus.cc:348] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.526018 28655 raft_consensus.cc:374] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.526036 28655 raft_consensus.cc:729] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.526072 28655 consensus_queue.cc:260] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.526333 28655 ts_tablet_manager.cc:1398] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:15.526392 28655 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.526994 28655 tablet_bootstrap.cc:654] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.527652 28655 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.527693 28655 ts_tablet_manager.cc:1367] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.527788 28655 raft_consensus.cc:348] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.527832 28655 raft_consensus.cc:374] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.527848 28655 raft_consensus.cc:729] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.527885 28655 consensus_queue.cc:260] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.527963 28655 ts_tablet_manager.cc:1398] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.528041 28655 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.528343 28655 tablet_bootstrap.cc:654] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.528789 27848 raft_consensus.cc:482] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.528839 27848 raft_consensus.cc:504] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.528945 27848 leader_election.cc:288] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.529126 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48a9db1d1d4c129f07e4e0eb9918ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.529192 27722 raft_consensus.cc:2454] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.529274 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48a9db1d1d4c129f07e4e0eb9918ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.529346 27449 raft_consensus.cc:2454] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.529337 27523 leader_election.cc:302] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.529420 27848 raft_consensus.cc:2781] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.529455 27848 raft_consensus.cc:482] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.529471 27848 raft_consensus.cc:3037] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.529974 27848 raft_consensus.cc:504] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.530071 27848 leader_election.cc:288] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.530186 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48a9db1d1d4c129f07e4e0eb9918ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.530244 27722 raft_consensus.cc:3037] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.530216 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec48a9db1d1d4c129f07e4e0eb9918ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.530273 27449 raft_consensus.cc:3037] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.539705 27453 raft_consensus.cc:1261] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.539713 27587 raft_consensus.cc:1261] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.539891 28774 consensus_queue.cc:1035] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.539966 27853 consensus_queue.cc:1035] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.541013 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.541157 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.541436 28664 tablet_bootstrap.cc:492] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.541481 28664 ts_tablet_manager.cc:1367] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20240503 02:09:15.541601 28664 raft_consensus.cc:348] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.541647 28664 raft_consensus.cc:374] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.541664 28664 raft_consensus.cc:729] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.541708 28664 consensus_queue.cc:260] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.541797 28664 ts_tablet_manager.cc:1398] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.541944 28706 tablet.cc:1722] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.542318 28664 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.542531 27722 raft_consensus.cc:2454] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.542604 28698 tablet.cc:1722] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.542641 28664 tablet_bootstrap.cc:654] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.542883 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:15.543530 27523 leader_election.cc:302] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.543684 27848 raft_consensus.cc:2781] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.543726 27848 raft_consensus.cc:686] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.543759 27848 consensus_queue.cc:237] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.544757 27198 catalog_manager.cc:5484] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.544862 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d] alter complete (version 1)
I20240503 02:09:15.545562 28685 raft_consensus.cc:482] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.545614 28685 raft_consensus.cc:504] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.545717 28685 leader_election.cc:288] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.545830 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766a17e99ae64b878920d451cc814a3c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.545890 27722 raft_consensus.cc:2454] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.545944 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766a17e99ae64b878920d451cc814a3c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.546002 27388 leader_election.cc:302] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.546015 27587 raft_consensus.cc:2454] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.546074 28685 raft_consensus.cc:2781] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.546106 28685 raft_consensus.cc:482] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.546123 28685 raft_consensus.cc:3037] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.546465 28664 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.546506 28664 ts_tablet_manager.cc:1367] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.546619 28664 raft_consensus.cc:348] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.546672 28664 raft_consensus.cc:374] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.546691 28664 raft_consensus.cc:729] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.546723 28664 consensus_queue.cc:260] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.546801 28664 ts_tablet_manager.cc:1398] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.546883 28664 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.547209 28664 tablet_bootstrap.cc:654] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.547863 28664 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.547909 28664 ts_tablet_manager.cc:1367] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.548029 28664 raft_consensus.cc:348] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.548089 28664 raft_consensus.cc:374] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.548107 28664 raft_consensus.cc:729] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.548135 28664 consensus_queue.cc:260] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.548234 28664 ts_tablet_manager.cc:1398] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.549350 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.549445 28655 tablet_bootstrap.cc:492] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.549492 28655 ts_tablet_manager.cc:1367] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20240503 02:09:15.549631 28655 raft_consensus.cc:348] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.549677 28655 raft_consensus.cc:374] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.549693 28655 raft_consensus.cc:729] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.549732 28655 consensus_queue.cc:260] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.549818 28655 ts_tablet_manager.cc:1398] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.549902 27449 raft_consensus.cc:2454] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.550436 28685 raft_consensus.cc:504] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.550546 28685 leader_election.cc:288] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.550670 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766a17e99ae64b878920d451cc814a3c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.550732 27587 raft_consensus.cc:3037] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.550729 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766a17e99ae64b878920d451cc814a3c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.550773 27722 raft_consensus.cc:3037] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.550969 28655 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.551309 28655 tablet_bootstrap.cc:654] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.551342 27587 raft_consensus.cc:2454] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.551342 27722 raft_consensus.cc:2454] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.551506 27386 leader_election.cc:302] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.551529 28686 tablet.cc:1722] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.551610 28409 raft_consensus.cc:2781] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.551646 28409 raft_consensus.cc:686] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.551681 28409 consensus_queue.cc:237] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.552819 27198 catalog_manager.cc:5484] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.553108 28655 tablet_bootstrap.cc:492] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.553155 28655 ts_tablet_manager.cc:1367] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.553269 28655 raft_consensus.cc:348] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.553342 28655 raft_consensus.cc:374] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.553361 28655 raft_consensus.cc:729] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.553401 28655 consensus_queue.cc:260] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.553480 28655 ts_tablet_manager.cc:1398] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.554420 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.568931 28693 raft_consensus.cc:482] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.568995 28693 raft_consensus.cc:504] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.569108 28693 leader_election.cc:288] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.569213 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6244d9ac8b40aab5f6e2a40cda8463" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.569270 27453 raft_consensus.cc:2454] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.569249 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6244d9ac8b40aab5f6e2a40cda8463" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.569307 27587 raft_consensus.cc:2454] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.569423 27659 leader_election.cc:302] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.569517 28693 raft_consensus.cc:2781] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.569552 28693 raft_consensus.cc:482] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.569566 28693 raft_consensus.cc:3037] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.570035 28693 raft_consensus.cc:504] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.570143 28693 leader_election.cc:288] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.570245 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6244d9ac8b40aab5f6e2a40cda8463" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.570256 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6244d9ac8b40aab5f6e2a40cda8463" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.570300 27453 raft_consensus.cc:3037] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.570313 27587 raft_consensus.cc:3037] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.570737 27453 raft_consensus.cc:2454] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.570873 27587 raft_consensus.cc:2454] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.570974 27659 leader_election.cc:302] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.571048 28693 raft_consensus.cc:2781] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.571094 28693 raft_consensus.cc:686] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.571135 28693 consensus_queue.cc:237] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.571995 27198 catalog_manager.cc:5484] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.572723 28693 consensus_queue.cc:1035] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.572856 28410 raft_consensus.cc:482] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.572903 28410 raft_consensus.cc:504] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.572999 28410 leader_election.cc:288] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.573127 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3900c1017d114230bc93d16ece225e3f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.573181 27722 raft_consensus.cc:2454] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.573432 27523 leader_election.cc:302] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.573565 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3900c1017d114230bc93d16ece225e3f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.573622 27453 raft_consensus.cc:2454] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.573810 28410 raft_consensus.cc:2781] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.573856 28410 raft_consensus.cc:482] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.573874 28410 raft_consensus.cc:3037] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.574426 28410 raft_consensus.cc:504] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.574517 28410 leader_election.cc:288] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.574673 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3900c1017d114230bc93d16ece225e3f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.574674 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3900c1017d114230bc93d16ece225e3f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.574731 27722 raft_consensus.cc:3037] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.574731 27453 raft_consensus.cc:3037] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.575277 27453 raft_consensus.cc:2454] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.575325 27722 raft_consensus.cc:2454] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.575464 27523 leader_election.cc:302] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.575500 28740 consensus_queue.cc:1035] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.575542 28410 raft_consensus.cc:2781] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.575592 28410 raft_consensus.cc:686] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.575625 28410 consensus_queue.cc:237] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.576426 27198 catalog_manager.cc:5484] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.576519 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:15.577029 28740 raft_consensus.cc:482] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.577077 28740 raft_consensus.cc:504] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.577191 28740 leader_election.cc:288] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.577391 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327dc0c3209f454499caeb31cdbe3f5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.577456 27582 raft_consensus.cc:2454] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.577596 27656 leader_election.cc:302] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.577764 28740 raft_consensus.cc:2781] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.577767 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327dc0c3209f454499caeb31cdbe3f5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.577804 28740 raft_consensus.cc:482] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.577822 28740 raft_consensus.cc:3037] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.577822 27449 raft_consensus.cc:2454] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.578267 28740 raft_consensus.cc:504] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.578367 28740 leader_election.cc:288] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.578518 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327dc0c3209f454499caeb31cdbe3f5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.578548 27848 raft_consensus.cc:482] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.578577 27582 raft_consensus.cc:3037] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.578598 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327dc0c3209f454499caeb31cdbe3f5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.578598 27848 raft_consensus.cc:504] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.578642 27449 raft_consensus.cc:3037] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.578687 27848 leader_election.cc:288] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.579154 27449 raft_consensus.cc:2454] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.579154 27582 raft_consensus.cc:2454] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.579298 27659 leader_election.cc:302] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.579365 28693 raft_consensus.cc:2781] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.579398 28693 raft_consensus.cc:686] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.579429 28693 consensus_queue.cc:237] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.579630 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3152be657da44e3ae7fd9501842c6ac" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.579684 27722 raft_consensus.cc:2454] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.579807 27523 leader_election.cc:302] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.579877 28410 raft_consensus.cc:2781] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.579912 28410 raft_consensus.cc:482] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.579928 28410 raft_consensus.cc:3037] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.579944 27722 raft_consensus.cc:1261] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.580147 27848 consensus_queue.cc:1035] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.580467 28410 raft_consensus.cc:504] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.580578 28410 leader_election.cc:288] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.580821 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3152be657da44e3ae7fd9501842c6ac" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.580909 27451 raft_consensus.cc:3037] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.581094 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3152be657da44e3ae7fd9501842c6ac" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.581290 27453 raft_consensus.cc:1261] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.581512 27451 raft_consensus.cc:2454] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.581491 28410 consensus_queue.cc:1035] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.581481 27194 catalog_manager.cc:5484] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.581574 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:15.581686 27524 leader_election.cc:302] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.581809 28410 raft_consensus.cc:2781] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.581851 28410 raft_consensus.cc:686] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.581885 28410 consensus_queue.cc:237] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.582322 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3152be657da44e3ae7fd9501842c6ac" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.582381 27722 raft_consensus.cc:3037] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.582473 28698 tablet.cc:1722] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.582937 27722 raft_consensus.cc:2454] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.583504 27194 catalog_manager.cc:5484] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.583680 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:15.584491 28706 tablet.cc:1722] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.584720 29090 mvcc.cc:205] Tried to move back new op lower bound from 7023420029253451776 to 7023420029238013952. Current Snapshot: MvccSnapshot[applied={T|T < 7023420029253451776}]
I20240503 02:09:15.584765 28686 tablet.cc:1722] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.587894 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03] alter complete (version 1)
I20240503 02:09:15.594602 27848 raft_consensus.cc:482] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.594650 27848 raft_consensus.cc:504] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.594748 27848 leader_election.cc:288] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.594872 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b675ecf68b049edac996e4fc7b0bbd8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.594861 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b675ecf68b049edac996e4fc7b0bbd8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.594941 27451 raft_consensus.cc:2454] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.594941 27722 raft_consensus.cc:2454] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.595053 27523 leader_election.cc:302] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.595120 28767 consensus_queue.cc:1035] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.595221 27848 raft_consensus.cc:2781] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.595258 27848 raft_consensus.cc:482] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.595274 27848 raft_consensus.cc:3037] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.595824 27848 raft_consensus.cc:504] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.595919 27848 leader_election.cc:288] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.596062 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b675ecf68b049edac996e4fc7b0bbd8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.596083 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b675ecf68b049edac996e4fc7b0bbd8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.596125 27722 raft_consensus.cc:3037] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.596138 27451 raft_consensus.cc:3037] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.596629 27722 raft_consensus.cc:2454] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.596637 27451 raft_consensus.cc:2454] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.596783 27523 leader_election.cc:302] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.596994 27848 raft_consensus.cc:2781] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.597043 27848 raft_consensus.cc:686] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.597075 27848 consensus_queue.cc:237] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.597273 28766 consensus_queue.cc:1035] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.598111 27197 catalog_manager.cc:5484] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.598978 27194 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2f1d3f2362414eeea600076e4ff8de6a" } modify_external_catalogs: true
I20240503 02:09:15.599098 27194 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2f1d3f2362414eeea600076e4ff8de6a" } modify_external_catalogs: true
I20240503 02:09:15.600364 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 11b157b6f5d04c31ac019de2d800d112
I20240503 02:09:15.600654 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b157b6f5d04c31ac019de2d800d112 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.600705 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b157b6f5d04c31ac019de2d800d112 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.600769 28661 tablet_replica.cc:330] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.600781 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 11b157b6f5d04c31ac019de2d800d112 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.600838 28661 raft_consensus.cc:2229] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.600859 28653 tablet_replica.cc:330] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.600889 28661 raft_consensus.cc:2258] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.600925 28653 raft_consensus.cc:2229] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.600952 28654 tablet_replica.cc:330] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.600965 28653 raft_consensus.cc:2258] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.601006 28654 raft_consensus.cc:2229] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.601059 28654 raft_consensus.cc:2258] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.601300 28653 ts_tablet_manager.cc:1875] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.601294 28654 ts_tablet_manager.cc:1875] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.602344 28661 ts_tablet_manager.cc:1875] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.602409 28654 ts_tablet_manager.cc:1888] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.602417 28653 ts_tablet_manager.cc:1888] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.602448 28654 log.cc:1196] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/11b157b6f5d04c31ac019de2d800d112
I20240503 02:09:15.602458 28653 log.cc:1196] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/11b157b6f5d04c31ac019de2d800d112
I20240503 02:09:15.602597 28653 ts_tablet_manager.cc:1909] T 11b157b6f5d04c31ac019de2d800d112 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.602634 28654 ts_tablet_manager.cc:1909] T 11b157b6f5d04c31ac019de2d800d112 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.603140 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 11b157b6f5d04c31ac019de2d800d112 (table default.Table_2f1d3f2362414eeea600076e4ff8de6a [id=9205c1ddc30a4442aa89c0b58b5b7700]) successfully deleted
I20240503 02:09:15.603164 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 11b157b6f5d04c31ac019de2d800d112 (table default.Table_2f1d3f2362414eeea600076e4ff8de6a [id=9205c1ddc30a4442aa89c0b58b5b7700]) successfully deleted
I20240503 02:09:15.603565 28661 ts_tablet_manager.cc:1888] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.603608 28661 log.cc:1196] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/11b157b6f5d04c31ac019de2d800d112
I20240503 02:09:15.603727 28661 ts_tablet_manager.cc:1909] T 11b157b6f5d04c31ac019de2d800d112 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.604182 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 11b157b6f5d04c31ac019de2d800d112 (table default.Table_2f1d3f2362414eeea600076e4ff8de6a [id=9205c1ddc30a4442aa89c0b58b5b7700]) successfully deleted
I20240503 02:09:15.611514 27848 raft_consensus.cc:482] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.611564 27848 raft_consensus.cc:504] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.611665 27848 leader_election.cc:288] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.611783 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f8911e604043c8a5b313ef62d206f0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.611789 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f8911e604043c8a5b313ef62d206f0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.611850 27722 raft_consensus.cc:2454] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.611850 27451 raft_consensus.cc:2454] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.611958 27523 leader_election.cc:302] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.612046 27848 raft_consensus.cc:2781] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.612076 27848 raft_consensus.cc:482] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.612092 27848 raft_consensus.cc:3037] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.612546 27848 raft_consensus.cc:504] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.612648 27848 leader_election.cc:288] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.612778 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f8911e604043c8a5b313ef62d206f0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.612779 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9f8911e604043c8a5b313ef62d206f0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.612833 27722 raft_consensus.cc:3037] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.612833 27451 raft_consensus.cc:3037] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.613237 27722 raft_consensus.cc:2454] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.613240 27451 raft_consensus.cc:2454] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.613379 27524 leader_election.cc:302] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.613471 27848 raft_consensus.cc:2781] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.613517 27848 raft_consensus.cc:686] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.613552 27848 consensus_queue.cc:237] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.614156 27194 catalog_manager.cc:5484] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.614253 27194 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.617488 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2949eee1a0ab47ca85d18131e12e8ab8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.617592 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.617846 28685 raft_consensus.cc:482] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.617898 28685 raft_consensus.cc:504] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.618009 28685 leader_election.cc:288] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.618090 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_fc4b588f9be84f7cb4257dbbe283bf28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.618144 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4021dcc28f40fdb505f2491e034a40" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
W20240503 02:09:15.618184 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc4b588f9be84f7cb4257dbbe283bf28 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.618198 27722 raft_consensus.cc:2454] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.618429 27388 leader_election.cc:302] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.618507 28685 raft_consensus.cc:2781] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.618541 28685 raft_consensus.cc:482] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.618557 28685 raft_consensus.cc:3037] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.618557 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4021dcc28f40fdb505f2491e034a40" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.618623 27582 raft_consensus.cc:2454] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.619145 28685 raft_consensus.cc:504] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.619241 28685 leader_election.cc:288] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.619364 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4021dcc28f40fdb505f2491e034a40" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.619421 27722 raft_consensus.cc:3037] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.619915 27722 raft_consensus.cc:2454] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.620054 27388 leader_election.cc:302] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.620157 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4021dcc28f40fdb505f2491e034a40" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.620211 27582 raft_consensus.cc:3037] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.620512 27693 tablet_service.cc:1467] Processing CreateTablet for tablet f2a07194e9874f7f9df96fc754e25779 (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.620613 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a07194e9874f7f9df96fc754e25779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.620640 27557 tablet_service.cc:1467] Processing CreateTablet for tablet f2a07194e9874f7f9df96fc754e25779 (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.620644 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_df5d4f768010413ab307211f72391619" } modify_external_catalogs: true
I20240503 02:09:15.620695 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_df5d4f768010413ab307211f72391619" } modify_external_catalogs: true
I20240503 02:09:15.620733 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a07194e9874f7f9df96fc754e25779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.620786 27582 raft_consensus.cc:2454] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.621165 27433 tablet_service.cc:1467] Processing CreateTablet for tablet f2a07194e9874f7f9df96fc754e25779 (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.621256 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a07194e9874f7f9df96fc754e25779. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.621637 28664 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.621932 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b4b67bcbc8de4b17a042600c659d7c34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.622017 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4b67bcbc8de4b17a042600c659d7c34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.622035 28664 tablet_bootstrap.cc:654] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.622082 28685 raft_consensus.cc:2781] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.622133 28685 raft_consensus.cc:686] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.622171 28685 consensus_queue.cc:237] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.622344 28656 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.622360 28655 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.622707 28656 tablet_bootstrap.cc:654] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.622795 28655 tablet_bootstrap.cc:654] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.623304 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ca84293d2cac408591b688cc2b45a5f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.623415 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca84293d2cac408591b688cc2b45a5f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.623814 28664 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.623860 28664 ts_tablet_manager.cc:1367] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.623874 28655 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.623914 28655 ts_tablet_manager.cc:1367] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.624007 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 5ce6b30fc5c742579f061c4bc50ecdf9
I20240503 02:09:15.624002 28664 raft_consensus.cc:348] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.624069 28664 raft_consensus.cc:374] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.624087 28664 raft_consensus.cc:729] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.624055 28655 raft_consensus.cc:348] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.624105 28655 raft_consensus.cc:374] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.624125 28655 raft_consensus.cc:729] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.624121 28664 consensus_queue.cc:260] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.624159 28655 consensus_queue.cc:260] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.624226 28664 ts_tablet_manager.cc:1398] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.624254 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.624584 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.624617 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 3630cebdb1b34cebab8118c250543ade (DEFAULT_TABLE table=default.Table_fc4b588f9be84f7cb4257dbbe283bf28 [id=63a30c6bc4fe4da2a9de3c3dad5dbcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.625051 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce6b30fc5c742579f061c4bc50ecdf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.625173 28661 tablet_replica.cc:330] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.625180 28654 tablet_replica.cc:330] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.625224 28661 raft_consensus.cc:2229] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.625250 28661 raft_consensus.cc:2258] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.625262 28654 raft_consensus.cc:2229] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.625305 28654 pending_rounds.cc:62] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:15.625339 28654 pending_rounds.cc:68] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420027902238720 op_type: NO_OP noop_request { }
I20240503 02:09:15.625370 28654 raft_consensus.cc:2866] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.625381 28661 ts_tablet_manager.cc:1875] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.625389 28654 raft_consensus.cc:2258] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.625914 27196 catalog_manager.cc:5484] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.626305 28654 ts_tablet_manager.cc:1875] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.626541 28661 ts_tablet_manager.cc:1888] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.626591 28661 log.cc:1196] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/5ce6b30fc5c742579f061c4bc50ecdf9
I20240503 02:09:15.626737 28661 ts_tablet_manager.cc:1909] T 5ce6b30fc5c742579f061c4bc50ecdf9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.626896 27848 raft_consensus.cc:482] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.626955 27848 raft_consensus.cc:504] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.627046 27848 leader_election.cc:288] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.627032 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 3630cebdb1b34cebab8118c250543ade (DEFAULT_TABLE table=default.Table_fc4b588f9be84f7cb4257dbbe283bf28 [id=63a30c6bc4fe4da2a9de3c3dad5dbcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.627110 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3630cebdb1b34cebab8118c250543ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.627236 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c08a7da01c54148bc19f3b29c00f841" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.627285 27722 raft_consensus.cc:2454] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.627490 27523 leader_election.cc:302] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.627632 27848 raft_consensus.cc:2781] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.627652 28654 ts_tablet_manager.cc:1888] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.627661 27848 raft_consensus.cc:482] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.627691 28654 log.cc:1196] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/5ce6b30fc5c742579f061c4bc50ecdf9
I20240503 02:09:15.627889 28654 ts_tablet_manager.cc:1909] T 5ce6b30fc5c742579f061c4bc50ecdf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.628391 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (table default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]) successfully deleted
I20240503 02:09:15.628409 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 3630cebdb1b34cebab8118c250543ade (DEFAULT_TABLE table=default.Table_fc4b588f9be84f7cb4257dbbe283bf28 [id=63a30c6bc4fe4da2a9de3c3dad5dbcc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.628497 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3630cebdb1b34cebab8118c250543ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.628638 28664 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.624258 28655 ts_tablet_manager.cc:1398] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.628959 28664 tablet_bootstrap.cc:654] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.629402 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3630cebdb1b34cebab8118c250543ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.629369 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c08a7da01c54148bc19f3b29c00f841" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.629446 27451 raft_consensus.cc:2454] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.630126 27703 tablet_service.cc:1467] Processing CreateTablet for tablet ff2349951a8b4f709892c40103b7cdf1 (DEFAULT_TABLE table=default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.630211 27565 tablet_service.cc:1467] Processing CreateTablet for tablet ff2349951a8b4f709892c40103b7cdf1 (DEFAULT_TABLE table=default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.630211 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab33852711d4212b1603fa04904cf0a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.630293 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2349951a8b4f709892c40103b7cdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.630389 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab33852711d4212b1603fa04904cf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.630399 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2349951a8b4f709892c40103b7cdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.631019 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab33852711d4212b1603fa04904cf0a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.631098 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ff2349951a8b4f709892c40103b7cdf1 (DEFAULT_TABLE table=default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.627684 27848 raft_consensus.cc:3037] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.631474 27194 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8f683e52b53441c48e52712f9271d74a" } modify_external_catalogs: true
I20240503 02:09:15.631513 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 1ab33852711d4212b1603fa04904cf0a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.631531 27194 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8f683e52b53441c48e52712f9271d74a" } modify_external_catalogs: true
I20240503 02:09:15.631587 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab33852711d4212b1603fa04904cf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.631935 27848 raft_consensus.cc:504] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.632040 27848 leader_election.cc:288] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.632229 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c08a7da01c54148bc19f3b29c00f841" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.632287 27722 raft_consensus.cc:3037] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.632668 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 36bb12209a3a42f092c724ba8373d79b
I20240503 02:09:15.632797 27722 raft_consensus.cc:2454] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.632985 28664 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.633045 28664 ts_tablet_manager.cc:1367] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.633082 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (table default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]) successfully deleted
I20240503 02:09:15.633164 28664 raft_consensus.cc:348] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.633232 28664 raft_consensus.cc:374] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.633250 28664 raft_consensus.cc:729] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.633355 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bb12209a3a42f092c724ba8373d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.633443 27523 leader_election.cc:302] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.633289 28664 consensus_queue.cc:260] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.633558 27848 raft_consensus.cc:2781] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.633597 27848 raft_consensus.cc:686] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.633594 28664 ts_tablet_manager.cc:1398] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.633653 28664 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.633631 27848 consensus_queue.cc:237] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.634014 28664 tablet_bootstrap.cc:654] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.634559 28664 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.634608 28664 ts_tablet_manager.cc:1367] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.634721 28664 raft_consensus.cc:348] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.634788 28664 raft_consensus.cc:374] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.634804 28664 raft_consensus.cc:729] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.634830 28664 consensus_queue.cc:260] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.634899 28664 ts_tablet_manager.cc:1398] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.634943 28664 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.635097 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c08a7da01c54148bc19f3b29c00f841" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.635157 27451 raft_consensus.cc:3037] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.635252 28664 tablet_bootstrap.cc:654] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.635562 27451 raft_consensus.cc:2454] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.635727 28664 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.635761 28664 ts_tablet_manager.cc:1367] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.636185 28664 raft_consensus.cc:348] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.636247 28664 raft_consensus.cc:374] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.636265 28664 raft_consensus.cc:729] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.636312 28664 consensus_queue.cc:260] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.636423 28664 ts_tablet_manager.cc:1398] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.637094 28655 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.637223 28653 tablet_replica.cc:330] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.637281 28653 raft_consensus.cc:2229] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.637311 28653 raft_consensus.cc:2258] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.637380 28655 tablet_bootstrap.cc:654] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.637432 28653 ts_tablet_manager.cc:1875] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.638612 28653 ts_tablet_manager.cc:1888] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.638645 28653 log.cc:1196] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/5ce6b30fc5c742579f061c4bc50ecdf9
I20240503 02:09:15.638743 28653 ts_tablet_manager.cc:1909] T 5ce6b30fc5c742579f061c4bc50ecdf9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.638844 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ab33852711d4212b1603fa04904cf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.638877 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2349951a8b4f709892c40103b7cdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.639626 28653 tablet_replica.cc:330] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.639685 28653 raft_consensus.cc:2229] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.639711 28653 raft_consensus.cc:2258] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.639972 28653 ts_tablet_manager.cc:1875] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.640066 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 5ce6b30fc5c742579f061c4bc50ecdf9 (table default.Table_df5d4f768010413ab307211f72391619 [id=fffd26501cf94eae8cfdb98f0884388b]) successfully deleted
I20240503 02:09:15.640264 27194 catalog_manager.cc:5484] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.640585 28655 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.640615 28655 ts_tablet_manager.cc:1367] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.640694 28655 raft_consensus.cc:348] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.640739 28655 raft_consensus.cc:374] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.640755 28655 raft_consensus.cc:729] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.640789 28655 consensus_queue.cc:260] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.640874 28655 ts_tablet_manager.cc:1398] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.640916 28655 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.641112 28653 ts_tablet_manager.cc:1888] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.641149 28653 log.cc:1196] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/36bb12209a3a42f092c724ba8373d79b
I20240503 02:09:15.641546 28653 ts_tablet_manager.cc:1909] T 36bb12209a3a42f092c724ba8373d79b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.641690 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet be7a66cc87834e739e958a2caf73de0c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.641809 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bb12209a3a42f092c724ba8373d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.641817 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 36bb12209a3a42f092c724ba8373d79b (table default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]) successfully deleted
I20240503 02:09:15.641893 28654 tablet_replica.cc:330] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.641877 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dbc6d5013a2f4b98af21729fd9dbb4f3" } new_table_name: "default.Table_66c2965235094c56a59bfb004135fc7a" modify_external_catalogs: true
I20240503 02:09:15.641944 28654 raft_consensus.cc:2229] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.641975 28654 raft_consensus.cc:2258] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.642164 28654 ts_tablet_manager.cc:1875] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.642697 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0a52c142ef664c02989e9bfa34f3d2f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.642704 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_60fd4282c8764278a417ee4f30f5686b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.642841 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_7b6bac8ae6b04c248a79775c7570af75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.642827 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60fd4282c8764278a417ee4f30f5686b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:15.642927 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b6bac8ae6b04c248a79775c7570af75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.642817 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_6dea9657852d452dbf25d95e393bbd74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.643113 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6dea9657852d452dbf25d95e393bbd74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:15.642815 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a52c142ef664c02989e9bfa34f3d2f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.643296 28654 ts_tablet_manager.cc:1888] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.641202 28655 tablet_bootstrap.cc:654] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.643337 28654 log.cc:1196] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/36bb12209a3a42f092c724ba8373d79b
I20240503 02:09:15.643527 28654 ts_tablet_manager.cc:1909] T 36bb12209a3a42f092c724ba8373d79b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.643711 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bb12209a3a42f092c724ba8373d79b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.643784 28661 tablet_replica.cc:330] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.643827 28661 raft_consensus.cc:2229] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.643875 28661 raft_consensus.cc:2258] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.644054 28661 ts_tablet_manager.cc:1875] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.644232 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ffea94b33c194129ba6db6c5a2af129a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.644311 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ffea94b33c194129ba6db6c5a2af129a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.645082 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 36bb12209a3a42f092c724ba8373d79b (table default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]) successfully deleted
I20240503 02:09:15.645226 28661 ts_tablet_manager.cc:1888] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.645267 28661 log.cc:1196] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/36bb12209a3a42f092c724ba8373d79b
I20240503 02:09:15.645444 28661 ts_tablet_manager.cc:1909] T 36bb12209a3a42f092c724ba8373d79b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.645849 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 36bb12209a3a42f092c724ba8373d79b (table default.Table_8f683e52b53441c48e52712f9271d74a [id=f37a0dae33174c608ed844854dca8f02]) successfully deleted
I20240503 02:09:15.645924 28409 raft_consensus.cc:482] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.645984 28409 raft_consensus.cc:504] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.646093 28409 leader_election.cc:288] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.646214 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4b1db49a3d4c1085b24c95509fe6a6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.646277 27582 raft_consensus.cc:2454] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.646404 27386 leader_election.cc:302] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.646504 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4b1db49a3d4c1085b24c95509fe6a6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.646566 27722 raft_consensus.cc:2454] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.646631 28409 raft_consensus.cc:2781] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.646663 28409 raft_consensus.cc:482] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.646680 28409 raft_consensus.cc:3037] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.647259 28409 raft_consensus.cc:504] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.647293 28655 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.647333 28655 ts_tablet_manager.cc:1367] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:15.647357 28409 leader_election.cc:288] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.647404 28409 raft_consensus.cc:482] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.647439 28655 raft_consensus.cc:348] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.647503 28655 raft_consensus.cc:374] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.647537 28655 raft_consensus.cc:729] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.647576 28655 consensus_queue.cc:260] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.647550 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4b1db49a3d4c1085b24c95509fe6a6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.647621 27723 raft_consensus.cc:3037] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.647652 28655 ts_tablet_manager.cc:1398] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.647711 28655 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.648031 28655 tablet_bootstrap.cc:654] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.648020 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet 43e8dd055a1d4673811f6e775340f008 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.648214 27723 raft_consensus.cc:2454] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.648352 27388 leader_election.cc:302] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.648392 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4b1db49a3d4c1085b24c95509fe6a6" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.648416 28685 raft_consensus.cc:2781] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.648440 27582 raft_consensus.cc:3037] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.648447 28685 raft_consensus.cc:686] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.648481 28685 consensus_queue.cc:237] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.648977 28655 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.649014 28655 ts_tablet_manager.cc:1367] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.649019 27582 raft_consensus.cc:2454] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.649135 28655 raft_consensus.cc:348] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.649184 28655 raft_consensus.cc:374] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.649200 28655 raft_consensus.cc:729] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.649267 27722 raft_consensus.cc:1261] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.649405 28656 tablet_bootstrap.cc:492] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.649444 28656 ts_tablet_manager.cc:1367] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.000s
I20240503 02:09:15.649407 28655 consensus_queue.cc:260] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.649569 28656 raft_consensus.cc:348] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.649591 27848 consensus_queue.cc:1035] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.649623 28656 raft_consensus.cc:374] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.649644 28656 raft_consensus.cc:729] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.649679 28656 consensus_queue.cc:260] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.649705 27451 raft_consensus.cc:1261] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.649793 28656 ts_tablet_manager.cc:1398] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.649871 28656 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.650192 28656 tablet_bootstrap.cc:654] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.650759 28655 ts_tablet_manager.cc:1398] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.651283 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 791439779c5b4f5f922403340c4b7a6f (DEFAULT_TABLE table=default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.651345 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 458b6fac520a4691a9de9a76c0305353 (DEFAULT_TABLE table=default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.651378 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791439779c5b4f5f922403340c4b7a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.651474 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 0d4a1909677544549029137b79501235 (DEFAULT_TABLE table=default.Table_60fd4282c8764278a417ee4f30f5686b [id=3b9e6ecebf8b4abe9b44c085ac626a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.651564 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d4a1909677544549029137b79501235. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.651614 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.651682 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369e89bd1625453c8f78e99b6d40bd61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.651703 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458b6fac520a4691a9de9a76c0305353. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.652351 27848 consensus_queue.cc:1035] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.652666 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 0d4a1909677544549029137b79501235 (DEFAULT_TABLE table=default.Table_60fd4282c8764278a417ee4f30f5686b [id=3b9e6ecebf8b4abe9b44c085ac626a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.652746 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d4a1909677544549029137b79501235. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.652786 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 0d4a1909677544549029137b79501235 (DEFAULT_TABLE table=default.Table_60fd4282c8764278a417ee4f30f5686b [id=3b9e6ecebf8b4abe9b44c085ac626a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.652856 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d4a1909677544549029137b79501235. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.652900 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.652982 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369e89bd1625453c8f78e99b6d40bd61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.653162 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 791439779c5b4f5f922403340c4b7a6f (DEFAULT_TABLE table=default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.653252 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791439779c5b4f5f922403340c4b7a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.653515 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d3152be657da44e3ae7fd9501842c6ac (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:15.653795 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bc4cb4cfddac46e28864e11a79981490" } new_table_name: "default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7" modify_external_catalogs: true
I20240503 02:09:15.653828 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 458b6fac520a4691a9de9a76c0305353 (DEFAULT_TABLE table=default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.653899 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458b6fac520a4691a9de9a76c0305353. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.654417 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 791439779c5b4f5f922403340c4b7a6f (DEFAULT_TABLE table=default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.654511 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 791439779c5b4f5f922403340c4b7a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.654645 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 43e8dd055a1d4673811f6e775340f008
I20240503 02:09:15.654727 27194 catalog_manager.cc:6554] ReplaceTablet: tablet 43e8dd055a1d4673811f6e775340f008 deleted and replaced by tablet 09426f1e3db54724a043f24c40f0e9f1
I20240503 02:09:15.654793 28655 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.654911 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 458b6fac520a4691a9de9a76c0305353 (DEFAULT_TABLE table=default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.654987 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458b6fac520a4691a9de9a76c0305353. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.655201 28655 tablet_bootstrap.cc:654] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.655195 28664 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.655514 28664 tablet_bootstrap.cc:654] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.655725 27848 raft_consensus.cc:482] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.655769 27848 raft_consensus.cc:504] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.655833 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e8dd055a1d4673811f6e775340f008 with delete_type TABLET_DATA_DELETED (replaced by tablet 09426f1e3db54724a043f24c40f0e9f1) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.655855 27848 leader_election.cc:288] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.656010 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d556e0a8904b519fc0656e4692d4ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.656064 27721 raft_consensus.cc:2454] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.656147 28740 raft_consensus.cc:482] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.656160 27523 leader_election.cc:302] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.656179 28740 raft_consensus.cc:504] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.656216 27848 raft_consensus.cc:2781] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.656237 27848 raft_consensus.cc:482] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.656252 27848 raft_consensus.cc:3037] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.656265 28740 leader_election.cc:288] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.656342 27693 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e8dd055a1d4673811f6e775340f008 with delete_type TABLET_DATA_DELETED (replaced by tablet 09426f1e3db54724a043f24c40f0e9f1) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.656414 28661 tablet_replica.cc:330] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.656401 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.656452 27451 raft_consensus.cc:2454] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.656459 28661 raft_consensus.cc:2229] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.656492 28661 raft_consensus.cc:2258] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.656625 27659 leader_election.cc:302] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.656689 28740 raft_consensus.cc:2781] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.656713 28740 raft_consensus.cc:482] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.656718 28661 ts_tablet_manager.cc:1875] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.656832 27848 raft_consensus.cc:504] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.656939 27848 leader_election.cc:288] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.657416 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d556e0a8904b519fc0656e4692d4ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.657470 27721 raft_consensus.cc:3037] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.657886 28661 ts_tablet_manager.cc:1888] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.657868 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.657927 28661 log.cc:1196] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/43e8dd055a1d4673811f6e775340f008
W20240503 02:09:15.658025 27656 leader_election.cc:341] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.658057 27721 raft_consensus.cc:2454] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.658084 28661 ts_tablet_manager.cc:1909] T 43e8dd055a1d4673811f6e775340f008 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.656729 28740 raft_consensus.cc:3037] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.658233 27523 leader_election.cc:302] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.658385 28507 consensus_queue.cc:1035] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.658483 28507 raft_consensus.cc:2781] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.658526 28507 raft_consensus.cc:686] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.658558 28507 consensus_queue.cc:237] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.658589 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
I20240503 02:09:15.658710 28740 raft_consensus.cc:504] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.658811 28740 leader_election.cc:288] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.658880 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d556e0a8904b519fc0656e4692d4ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.658931 27453 raft_consensus.cc:2454] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.658986 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59d556e0a8904b519fc0656e4692d4ca" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.659019 27449 raft_consensus.cc:3037] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.658983 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_63ccfec42f064aacaf0c950ec3442742" } new_table_name: "default.Table_3acf1387019c4a17ba3e98e20e8a90d6" modify_external_catalogs: true
I20240503 02:09:15.647440 28409 raft_consensus.cc:504] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.659572 28409 leader_election.cc:288] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.659598 27449 raft_consensus.cc:2454] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.659742 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 43e8dd055a1d4673811f6e775340f008 with delete_type TABLET_DATA_DELETED (replaced by tablet 09426f1e3db54724a043f24c40f0e9f1) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.659804 28653 tablet_replica.cc:330] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.659795 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8419275f80940f4aad5ebc259cb52fc" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.659844 27723 raft_consensus.cc:2454] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.659847 28653 raft_consensus.cc:2229] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.659895 28653 raft_consensus.cc:2258] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.659905 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8419275f80940f4aad5ebc259cb52fc" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.659951 27582 raft_consensus.cc:2454] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.659996 27388 leader_election.cc:302] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.660068 28409 raft_consensus.cc:2781] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.660092 28409 raft_consensus.cc:482] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.660109 28409 raft_consensus.cc:3037] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.660149 28653 ts_tablet_manager.cc:1875] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.660552 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 (DEFAULT_TABLE table=default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.660550 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.660604 27449 raft_consensus.cc:3037] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.660637 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09426f1e3db54724a043f24c40f0e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.660647 28409 raft_consensus.cc:504] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.660753 28409 leader_election.cc:288] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.661124 27449 raft_consensus.cc:2454] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.661224 28653 ts_tablet_manager.cc:1888] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.661267 28653 log.cc:1196] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/43e8dd055a1d4673811f6e775340f008
I20240503 02:09:15.661264 27659 leader_election.cc:302] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.661391 27853 raft_consensus.cc:2781] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.661427 27853 raft_consensus.cc:686] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.661455 27853 consensus_queue.cc:237] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.661547 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8419275f80940f4aad5ebc259cb52fc" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.661599 27723 raft_consensus.cc:3037] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.661792 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_6f96d95848404e34a565d59b7e182cc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.661877 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f96d95848404e34a565d59b7e182cc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.662137 27723 raft_consensus.cc:2454] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.662248 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.662340 27388 leader_election.cc:302] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.662403 28409 raft_consensus.cc:2781] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.662431 28409 raft_consensus.cc:686] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.662463 28409 consensus_queue.cc:237] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.662456 27194 catalog_manager.cc:5484] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.662717 27198 catalog_manager.cc:5484] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.662799 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d3152be657da44e3ae7fd9501842c6ac (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:15.662853 27202 catalog_manager.cc:5484] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.662938 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.663134 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8419275f80940f4aad5ebc259cb52fc" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.663187 27582 raft_consensus.cc:3037] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.663731 27582 raft_consensus.cc:2454] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.664201 28698 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:15.664306 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 (DEFAULT_TABLE table=default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.664686 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 (DEFAULT_TABLE table=default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.664770 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09426f1e3db54724a043f24c40f0e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.665462 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3caec400749841ad8ba3c6822d3677ab" } new_table_name: "default.Table_7854e52face84af4ada30d9f568b9c84" modify_external_catalogs: true
W20240503 02:09:15.665954 27656 leader_election.cc:341] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:15.666049 27419 tablet_service.cc:1467] Processing CreateTablet for tablet c0d68fe045a046538ebc5b0ef87874ee (DEFAULT_TABLE table=default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.666132 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d68fe045a046538ebc5b0ef87874ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.666563 28698 tablet.cc:1722] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.667114 27565 tablet_service.cc:1467] Processing CreateTablet for tablet c0d68fe045a046538ebc5b0ef87874ee (DEFAULT_TABLE table=default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.667202 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d68fe045a046538ebc5b0ef87874ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.667210 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 369e89bd1625453c8f78e99b6d40bd61 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.667637 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.667721 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369e89bd1625453c8f78e99b6d40bd61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.667770 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 (DEFAULT_TABLE table=default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.667838 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c35a98c21b848e79a38f1b9cf5919a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.668835 27194 catalog_manager.cc:5484] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.669121 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09426f1e3db54724a043f24c40f0e9f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.669308 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 369e89bd1625453c8f78e99b6d40bd61
I20240503 02:09:15.669415 27202 catalog_manager.cc:6554] ReplaceTablet: tablet 369e89bd1625453c8f78e99b6d40bd61 deleted and replaced by tablet 138ffe8057904a2485289b61064bf382
I20240503 02:09:15.669405 27721 raft_consensus.cc:1261] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.669572 28409 consensus_queue.cc:1035] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.669661 28706 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:15.669752 27693 tablet_service.cc:1467] Processing CreateTablet for tablet c0d68fe045a046538ebc5b0ef87874ee (DEFAULT_TABLE table=default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.669838 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0d68fe045a046538ebc5b0ef87874ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.670522 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 43e8dd055a1d4673811f6e775340f008 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.670951 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 (DEFAULT_TABLE table=default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.671012 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841] alter complete (version 1)
I20240503 02:09:15.671031 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c35a98c21b848e79a38f1b9cf5919a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.671757 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.671890 28686 tablet.cc:1722] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20240503 02:09:15.672111 27189 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 369e89bd1625453c8f78e99b6d40bd61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369e89bd1625453c8f78e99b6d40bd61 already in progress: creating tablet
I20240503 02:09:15.672343 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.672348 28710 tablet.cc:1722] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.672439 27848 consensus_queue.cc:1035] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.672770 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3acf1387019c4a17ba3e98e20e8a90d6 [id=c46a0a1c65dd47e38382f2b2d19bff05] alter complete (version 3)
I20240503 02:09:15.673235 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffe8057904a2485289b61064bf382 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.673339 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffe8057904a2485289b61064bf382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.673722 28706 tablet.cc:1722] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.674021 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffe8057904a2485289b61064bf382 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.674070 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.674103 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffe8057904a2485289b61064bf382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.674253 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.674348 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ffea94b33c194129ba6db6c5a2af129a" } modify_external_catalogs: true
I20240503 02:09:15.674408 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ffea94b33c194129ba6db6c5a2af129a" } modify_external_catalogs: true
I20240503 02:09:15.674696 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_54e8f67133a5407b827dee1b3c2e0c23" } new_table_name: "default.Table_e7f335e7089a44c9aff8bc2ddf88f113" modify_external_catalogs: true
I20240503 02:09:15.675280 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_630051f741264c3a925573b248899ffd" } modify_external_catalogs: true
I20240503 02:09:15.675330 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_630051f741264c3a925573b248899ffd" } modify_external_catalogs: true
I20240503 02:09:15.675736 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c0d68fe045a046538ebc5b0ef87874ee
I20240503 02:09:15.676039 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.676060 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 138ffe8057904a2485289b61064bf382 (DEFAULT_TABLE table=default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.676136 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 138ffe8057904a2485289b61064bf382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.676349 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.677177 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet fcd5396b5f8041959f7e2ed462fbf825
W20240503 02:09:15.678260 27189 catalog_manager.cc:4797] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet c0d68fe045a046538ebc5b0ef87874ee because the tablet was not found. No further retry: Not found: Tablet not found: c0d68fe045a046538ebc5b0ef87874ee
I20240503 02:09:15.678287 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 43e8dd055a1d4673811f6e775340f008 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.678328 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.678397 28661 tablet_replica.cc:330] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.678431 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a] alter complete (version 1)
I20240503 02:09:15.678449 28661 raft_consensus.cc:2229] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.678478 28661 raft_consensus.cc:2258] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.678774 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.678963 28661 ts_tablet_manager.cc:1875] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.680261 28661 ts_tablet_manager.cc:1888] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.680326 28661 log.cc:1196] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fcd5396b5f8041959f7e2ed462fbf825
I20240503 02:09:15.680619 28661 ts_tablet_manager.cc:1909] T fcd5396b5f8041959f7e2ed462fbf825 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.680888 28656 tablet_bootstrap.cc:492] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.680931 28656 ts_tablet_manager.cc:1367] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.001s
I20240503 02:09:15.681059 28656 raft_consensus.cc:348] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.681108 28656 raft_consensus.cc:374] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.681125 28656 raft_consensus.cc:729] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.681161 28656 consensus_queue.cc:260] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.681249 28656 ts_tablet_manager.cc:1398] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.681339 28656 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.681703 28656 tablet_bootstrap.cc:654] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.684408 28710 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:15.684849 28767 consensus_queue.cc:1035] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.685019 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet fcd5396b5f8041959f7e2ed462fbf825 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.685384 28698 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:15.685452 27720 raft_consensus.cc:1261] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.685575 28767 consensus_queue.cc:1035] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.685834 28706 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:15.685979 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.686714 28664 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.686751 28664 ts_tablet_manager.cc:1367] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240503 02:09:15.686861 27587 raft_consensus.cc:1261] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.686872 28664 raft_consensus.cc:348] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.686923 28664 raft_consensus.cc:374] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.686939 28664 raft_consensus.cc:729] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.686980 28656 tablet_bootstrap.cc:492] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.686975 28664 consensus_queue.cc:260] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.687000 27582 raft_consensus.cc:1261] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.687031 28656 ts_tablet_manager.cc:1367] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:15.687089 28664 ts_tablet_manager.cc:1398] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.687137 28664 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.687157 28656 raft_consensus.cc:348] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.687202 28656 raft_consensus.cc:374] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.687219 28656 raft_consensus.cc:729] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.687258 28656 consensus_queue.cc:260] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.687338 28656 ts_tablet_manager.cc:1398] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.687383 28656 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.687378 28710 tablet.cc:1722] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.687438 28685 consensus_queue.cc:1035] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.687465 28409 consensus_queue.cc:1035] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.687513 28664 tablet_bootstrap.cc:654] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.687752 28656 tablet_bootstrap.cc:654] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.687863 28654 tablet_replica.cc:330] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.687918 28654 raft_consensus.cc:2229] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.687979 28654 raft_consensus.cc:2258] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.689118 29046 mvcc.cc:205] Tried to move back new op lower bound from 7023420029606039552 to 7023420028175601664. Current Snapshot: MvccSnapshot[applied={T|T < 7023420029606039552}]
I20240503 02:09:15.689184 28710 tablet.cc:1722] T 6b1e79e771df40f7abcdecaf683ae61b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:15.689349 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:15.690253 28507 raft_consensus.cc:482] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.690294 28507 raft_consensus.cc:504] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.690394 28507 leader_election.cc:288] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.690635 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb40e254ab40f09cc72093bf50c91b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.690711 27721 raft_consensus.cc:2454] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.690745 28694 tablet.cc:1722] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.691069 29165 tablet.cc:1722] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.691257 28664 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.691293 28664 ts_tablet_manager.cc:1367] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.691426 28664 raft_consensus.cc:348] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.691478 28664 raft_consensus.cc:374] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.691495 28664 raft_consensus.cc:729] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.691545 28664 consensus_queue.cc:260] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.691570 27523 leader_election.cc:302] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.691625 28664 ts_tablet_manager.cc:1398] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.691632 28507 raft_consensus.cc:2781] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.691653 28507 raft_consensus.cc:482] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.691670 28507 raft_consensus.cc:3037] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.691671 28664 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.691992 28664 tablet_bootstrap.cc:654] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.691984 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:15.692328 28507 raft_consensus.cc:504] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.692431 28507 leader_election.cc:288] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.692646 27853 consensus_queue.cc:1035] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.692821 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb40e254ab40f09cc72093bf50c91b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.692864 27720 raft_consensus.cc:3037] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.693792 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb40e254ab40f09cc72093bf50c91b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.693830 27451 raft_consensus.cc:1261] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.693853 27453 raft_consensus.cc:2454] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.692647 27721 raft_consensus.cc:1261] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.693975 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fcd5396b5f8041959f7e2ed462fbf825 (table default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]) successfully deleted
I20240503 02:09:15.694065 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fb40e254ab40f09cc72093bf50c91b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.694123 27450 raft_consensus.cc:3037] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.694562 28410 consensus_queue.cc:1035] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.694754 27450 raft_consensus.cc:2454] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.695580 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.695623 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:15.695897 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f] alter complete (version 1)
I20240503 02:09:15.696264 27720 raft_consensus.cc:2454] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.696449 28654 ts_tablet_manager.cc:1875] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.696482 28507 raft_consensus.cc:482] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.696512 28507 raft_consensus.cc:504] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.696583 28507 leader_election.cc:288] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.696583 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.696468 27523 leader_election.cc:302] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.696748 28507 raft_consensus.cc:2781] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.696780 28507 raft_consensus.cc:686] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.696812 28507 consensus_queue.cc:237] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.697041 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.697400 29165 tablet.cc:1722] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.697793 28654 ts_tablet_manager.cc:1888] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.697832 28654 log.cc:1196] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/43e8dd055a1d4673811f6e775340f008
I20240503 02:09:15.698014 28654 ts_tablet_manager.cc:1909] T 43e8dd055a1d4673811f6e775340f008 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.698562 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
W20240503 02:09:15.698735 27187 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 369e89bd1625453c8f78e99b6d40bd61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369e89bd1625453c8f78e99b6d40bd61 already in progress: creating tablet
W20240503 02:09:15.698872 27187 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet c0d68fe045a046538ebc5b0ef87874ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0d68fe045a046538ebc5b0ef87874ee already in progress: creating tablet
I20240503 02:09:15.698948 28654 tablet_replica.cc:330] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.699013 28654 raft_consensus.cc:2229] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.699057 28654 raft_consensus.cc:2258] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.699256 28654 ts_tablet_manager.cc:1875] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.699954 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.699946 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2349951a8b4f709892c40103b7cdf1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.700011 27450 raft_consensus.cc:2379] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:15.700313 27194 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_7854e52face84af4ada30d9f568b9c84 [id=b2a8365635134977a758514aa7a4351d] alter complete (version 4)
I20240503 02:09:15.700587 28410 consensus_queue.cc:1035] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.701122 28693 consensus_queue.cc:1035] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.701857 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.701905 27198 catalog_manager.cc:5484] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.702172 28654 ts_tablet_manager.cc:1888] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.702375 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.702214 28654 log.cc:1196] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fcd5396b5f8041959f7e2ed462fbf825
I20240503 02:09:15.702555 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 (DEFAULT_TABLE table=default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.702916 28654 ts_tablet_manager.cc:1909] T fcd5396b5f8041959f7e2ed462fbf825 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.703104 27524 leader_election.cc:302] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:15.703217 28410 raft_consensus.cc:3037] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.703469 28653 ts_tablet_manager.cc:1909] T 43e8dd055a1d4673811f6e775340f008 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.703516 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fcd5396b5f8041959f7e2ed462fbf825 (table default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]) successfully deleted
I20240503 02:09:15.703644 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c35a98c21b848e79a38f1b9cf5919a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.703734 28653 tablet_replica.cc:330] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.703763 28410 raft_consensus.cc:2726] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:15.703796 28653 raft_consensus.cc:2229] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.703823 28653 raft_consensus.cc:2258] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.704882 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 43e8dd055a1d4673811f6e775340f008 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
I20240503 02:09:15.705152 28710 tablet.cc:1722] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.705255 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:15.705401 28694 tablet.cc:1722] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.705657 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.705724 28661 tablet_replica.cc:330] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.705766 28661 raft_consensus.cc:2229] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.705793 28661 raft_consensus.cc:2258] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.705960 28661 ts_tablet_manager.cc:1875] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.707176 28661 ts_tablet_manager.cc:1888] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.707223 28661 log.cc:1196] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/369e89bd1625453c8f78e99b6d40bd61
I20240503 02:09:15.707367 28661 ts_tablet_manager.cc:1909] T 369e89bd1625453c8f78e99b6d40bd61 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.707641 28656 tablet_bootstrap.cc:492] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.707680 28656 ts_tablet_manager.cc:1367] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20240503 02:09:15.707803 28656 raft_consensus.cc:348] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.707849 28656 raft_consensus.cc:374] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.707864 28656 raft_consensus.cc:729] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.707885 28656 consensus_queue.cc:260] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.707953 28656 ts_tablet_manager.cc:1398] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.708010 28656 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.708202 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 369e89bd1625453c8f78e99b6d40bd61 (table default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:15.708352 28656 tablet_bootstrap.cc:654] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.710032 27719 raft_consensus.cc:1261] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.710189 27848 consensus_queue.cc:1035] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.710975 27450 raft_consensus.cc:1261] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.711079 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.711313 27848 consensus_queue.cc:1035] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:15.711277 29158 consensus_queue.cc:1159] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bd4b79bd8b57443faa6fa5e464570188" status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20240503 02:09:15.711551 28653 ts_tablet_manager.cc:1875] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.712208 28656 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.712247 28656 ts_tablet_manager.cc:1367] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.712366 28656 raft_consensus.cc:348] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.712416 28656 raft_consensus.cc:374] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.712433 28656 raft_consensus.cc:729] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.712469 28656 consensus_queue.cc:260] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.712574 28656 ts_tablet_manager.cc:1398] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.712599 29165 tablet.cc:1722] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:15.712625 28656 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.712850 28653 ts_tablet_manager.cc:1888] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.712909 28653 log.cc:1196] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fcd5396b5f8041959f7e2ed462fbf825
I20240503 02:09:15.712985 28656 tablet_bootstrap.cc:654] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.713007 28409 raft_consensus.cc:482] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.713049 28409 raft_consensus.cc:504] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.713086 28653 ts_tablet_manager.cc:1909] T fcd5396b5f8041959f7e2ed462fbf825 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.713150 28409 leader_election.cc:288] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.713372 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.713445 27582 raft_consensus.cc:2454] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.713476 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.713536 27721 raft_consensus.cc:2454] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.713616 27386 leader_election.cc:302] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.713696 28409 raft_consensus.cc:2781] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.713725 28409 raft_consensus.cc:482] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.713743 28409 raft_consensus.cc:3037] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.713948 28664 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.713989 28664 ts_tablet_manager.cc:1367] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.000s
I20240503 02:09:15.714109 28664 raft_consensus.cc:348] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.714164 28664 raft_consensus.cc:374] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.714181 28664 raft_consensus.cc:729] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.714213 28664 consensus_queue.cc:260] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.714253 27194 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.714300 28664 ts_tablet_manager.cc:1398] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.714335 28409 raft_consensus.cc:504] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.714365 28664 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.714432 28409 leader_election.cc:288] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.714659 27721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.714704 28664 tablet_bootstrap.cc:654] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.714718 27721 raft_consensus.cc:3037] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.715062 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c89b472abbe4fceb027714afa92a7b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.715116 27582 raft_consensus.cc:3037] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.715325 27721 raft_consensus.cc:2454] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.715453 27388 leader_election.cc:302] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.715533 28409 raft_consensus.cc:2781] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.715565 28409 raft_consensus.cc:686] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.715598 28409 consensus_queue.cc:237] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.715705 27582 raft_consensus.cc:2454] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.716367 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fcd5396b5f8041959f7e2ed462fbf825 (table default.Table_630051f741264c3a925573b248899ffd [id=fd6043e45037461fbd4a34cbe357eb6c]) successfully deleted
W20240503 02:09:15.716444 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet c0d68fe045a046538ebc5b0ef87874ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c0d68fe045a046538ebc5b0ef87874ee already in progress: creating tablet
W20240503 02:09:15.716507 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 369e89bd1625453c8f78e99b6d40bd61 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 369e89bd1625453c8f78e99b6d40bd61 already in progress: creating tablet
I20240503 02:09:15.717711 27194 catalog_manager.cc:5484] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.717994 28706 tablet.cc:1722] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.718811 29172 tablet.cc:1722] T 59d556e0a8904b519fc0656e4692d4ca P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.718818 28767 consensus_queue.cc:1035] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.719410 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 59d556e0a8904b519fc0656e4692d4ca (table default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481]). Expected version 1 got 0
I20240503 02:09:15.721120 29167 tablet.cc:1722] T 59d556e0a8904b519fc0656e4692d4ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.722045 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_2c38711e62194693a4f5e082589671f1 [id=88e8538ff21849f2af7f32673e4fe481] alter complete (version 1)
I20240503 02:09:15.723290 28664 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.723335 28664 ts_tablet_manager.cc:1367] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:15.723462 28664 raft_consensus.cc:348] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.723532 28664 raft_consensus.cc:374] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.723551 28664 raft_consensus.cc:729] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.723558 28656 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.723585 28664 consensus_queue.cc:260] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.723605 28656 ts_tablet_manager.cc:1367] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:15.723670 28664 ts_tablet_manager.cc:1398] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.723733 28664 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.723726 28656 raft_consensus.cc:348] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.723778 28656 raft_consensus.cc:374] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.723795 28656 raft_consensus.cc:729] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.723827 28656 consensus_queue.cc:260] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.723903 28656 ts_tablet_manager.cc:1398] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.723953 28656 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.724193 28664 tablet_bootstrap.cc:654] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.724195 28710 tablet.cc:1722] T 59d556e0a8904b519fc0656e4692d4ca P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.724267 28656 tablet_bootstrap.cc:654] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.724746 28656 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.724787 28656 ts_tablet_manager.cc:1367] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.724905 28656 raft_consensus.cc:348] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.724956 28656 raft_consensus.cc:374] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.724972 28656 raft_consensus.cc:729] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.725011 28656 consensus_queue.cc:260] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.725085 28656 ts_tablet_manager.cc:1398] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.725134 28656 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.725497 28656 tablet_bootstrap.cc:654] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.725466 28409 consensus_queue.cc:1035] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.725579 28410 consensus_queue.cc:1035] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.727957 28655 tablet_bootstrap.cc:492] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.728009 28655 ts_tablet_manager.cc:1367] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.073s user 0.000s sys 0.001s
I20240503 02:09:15.728108 28655 raft_consensus.cc:348] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.728152 28655 raft_consensus.cc:374] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.728176 28655 raft_consensus.cc:729] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.728236 28655 consensus_queue.cc:260] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.728276 28664 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.728318 28664 ts_tablet_manager.cc:1367] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.728324 28655 ts_tablet_manager.cc:1398] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.728431 28664 raft_consensus.cc:348] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.728480 28664 raft_consensus.cc:374] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.728505 28664 raft_consensus.cc:729] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.728523 28655 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.728538 28664 consensus_queue.cc:260] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.728639 28664 ts_tablet_manager.cc:1398] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.728778 28655 tablet_bootstrap.cc:654] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.728883 28664 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.729243 28664 tablet_bootstrap.cc:654] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.730814 28507 consensus_queue.cc:1035] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.731534 28664 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.731578 28664 ts_tablet_manager.cc:1367] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:15.731628 28656 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.731678 28656 ts_tablet_manager.cc:1367] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:15.731716 28664 raft_consensus.cc:348] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.731768 28664 raft_consensus.cc:374] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.731789 28664 raft_consensus.cc:729] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.731814 28664 consensus_queue.cc:260] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.731839 28656 raft_consensus.cc:348] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.731901 28656 raft_consensus.cc:374] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.731925 28656 raft_consensus.cc:729] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.731964 28656 consensus_queue.cc:260] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.732054 28656 ts_tablet_manager.cc:1398] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.732182 28656 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.732472 28664 ts_tablet_manager.cc:1398] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.732542 28664 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.732590 28656 tablet_bootstrap.cc:654] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.732875 28664 tablet_bootstrap.cc:654] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.732846 28409 consensus_queue.cc:1035] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.733068 28655 tablet_bootstrap.cc:492] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.733107 28655 ts_tablet_manager.cc:1367] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.733197 28655 raft_consensus.cc:348] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.733230 28656 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.733253 28655 raft_consensus.cc:374] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.733261 28656 ts_tablet_manager.cc:1367] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.733273 28655 raft_consensus.cc:729] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.733309 28655 consensus_queue.cc:260] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.733392 28656 raft_consensus.cc:348] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.733462 28655 ts_tablet_manager.cc:1398] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.733522 28655 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.733755 28655 tablet_bootstrap.cc:654] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.733456 28656 raft_consensus.cc:374] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.734041 28656 raft_consensus.cc:729] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.734076 28656 consensus_queue.cc:260] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.734190 28656 ts_tablet_manager.cc:1398] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.734237 28656 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.734686 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.734956 28654 tablet_replica.cc:330] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.735011 28654 raft_consensus.cc:2229] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.735037 28654 raft_consensus.cc:2258] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.735244 28656 tablet_bootstrap.cc:654] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.735399 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
W20240503 02:09:15.735467 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 09426f1e3db54724a043f24c40f0e9f1 consensus state has no leader
I20240503 02:09:15.735780 28656 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.735818 28656 ts_tablet_manager.cc:1367] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.735936 28656 raft_consensus.cc:348] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.735989 28656 raft_consensus.cc:374] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.736007 28656 raft_consensus.cc:729] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.736038 28656 consensus_queue.cc:260] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.736088 28664 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.736126 28664 ts_tablet_manager.cc:1367] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.736130 28656 ts_tablet_manager.cc:1398] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.736164 28654 ts_tablet_manager.cc:1875] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.736183 28656 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.736275 28664 raft_consensus.cc:348] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.736325 28664 raft_consensus.cc:374] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.736342 28664 raft_consensus.cc:729] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.736373 28664 consensus_queue.cc:260] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.736454 28664 ts_tablet_manager.cc:1398] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.736487 28656 tablet_bootstrap.cc:654] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.736521 28664 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.736924 28664 tablet_bootstrap.cc:654] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.737383 28656 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.737442 28656 ts_tablet_manager.cc:1367] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.737509 28654 ts_tablet_manager.cc:1888] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.737548 28654 log.cc:1196] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/369e89bd1625453c8f78e99b6d40bd61
I20240503 02:09:15.737560 28656 raft_consensus.cc:348] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.737609 28656 raft_consensus.cc:374] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.737627 28656 raft_consensus.cc:729] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.737661 28656 consensus_queue.cc:260] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.737690 28654 ts_tablet_manager.cc:1909] T 369e89bd1625453c8f78e99b6d40bd61 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.737735 28656 ts_tablet_manager.cc:1398] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.737778 28656 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.738034 28664 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.738073 28664 ts_tablet_manager.cc:1367] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.738189 28664 raft_consensus.cc:348] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.738227 28656 tablet_bootstrap.cc:654] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.738237 28664 raft_consensus.cc:374] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.738267 28664 raft_consensus.cc:729] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.738299 28664 consensus_queue.cc:260] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.738377 28664 ts_tablet_manager.cc:1398] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.738735 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.738780 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.738971 28654 tablet_replica.cc:330] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.738984 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 369e89bd1625453c8f78e99b6d40bd61 (table default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:15.739017 28654 raft_consensus.cc:2229] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.739040 28654 raft_consensus.cc:2258] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.739239 28661 tablet_replica.cc:330] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.739285 28654 ts_tablet_manager.cc:1875] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.739296 28661 raft_consensus.cc:2229] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.739326 28661 raft_consensus.cc:2258] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.739665 28661 ts_tablet_manager.cc:1875] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.739676 28655 tablet_bootstrap.cc:492] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.739715 28655 ts_tablet_manager.cc:1367] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:15.739841 28655 raft_consensus.cc:348] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.739902 28655 raft_consensus.cc:374] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.739920 28655 raft_consensus.cc:729] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.739954 28655 consensus_queue.cc:260] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.740031 28655 ts_tablet_manager.cc:1398] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.740095 28655 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.740388 28654 ts_tablet_manager.cc:1888] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.740428 28655 tablet_bootstrap.cc:654] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.740435 28654 log.cc:1196] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c0d68fe045a046538ebc5b0ef87874ee
I20240503 02:09:15.740823 28655 tablet_bootstrap.cc:492] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.740861 28655 ts_tablet_manager.cc:1367] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.740949 28655 raft_consensus.cc:348] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.740984 28661 ts_tablet_manager.cc:1888] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.741020 28661 log.cc:1196] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c0d68fe045a046538ebc5b0ef87874ee
I20240503 02:09:15.741007 28655 raft_consensus.cc:374] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.741044 28655 raft_consensus.cc:729] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.741044 28654 ts_tablet_manager.cc:1909] T c0d68fe045a046538ebc5b0ef87874ee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.741076 28655 consensus_queue.cc:260] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.741160 28655 ts_tablet_manager.cc:1398] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.741169 28661 ts_tablet_manager.cc:1909] T c0d68fe045a046538ebc5b0ef87874ee P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.741209 28655 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.741298 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c0d68fe045a046538ebc5b0ef87874ee (table default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]) successfully deleted
I20240503 02:09:15.741462 28655 tablet_bootstrap.cc:654] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.742074 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c0d68fe045a046538ebc5b0ef87874ee (table default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]) successfully deleted
I20240503 02:09:15.742202 28656 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.742223 28655 tablet_bootstrap.cc:492] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.742239 28656 ts_tablet_manager.cc:1367] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.742255 28655 ts_tablet_manager.cc:1367] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.742314 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.742354 28656 raft_consensus.cc:348] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.742373 28655 raft_consensus.cc:348] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.742406 28656 raft_consensus.cc:374] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.742422 28655 raft_consensus.cc:374] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.742424 28656 raft_consensus.cc:729] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.742441 28655 raft_consensus.cc:729] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.742460 28656 consensus_queue.cc:260] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.742475 28655 consensus_queue.cc:260] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.742539 28656 ts_tablet_manager.cc:1398] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.742578 28655 ts_tablet_manager.cc:1398] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.742637 28655 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.742975 28655 tablet_bootstrap.cc:654] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.743795 27194 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.744028 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.744221 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet c0d68fe045a046538ebc5b0ef87874ee with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.744292 28653 tablet_replica.cc:330] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.744334 28653 raft_consensus.cc:2229] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.744359 28653 raft_consensus.cc:2258] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.744489 28653 ts_tablet_manager.cc:1875] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.744693 28655 tablet_bootstrap.cc:492] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.744732 28655 ts_tablet_manager.cc:1367] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.744840 28655 raft_consensus.cc:348] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.744887 28655 raft_consensus.cc:374] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.744904 28655 raft_consensus.cc:729] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.744946 28655 consensus_queue.cc:260] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.745036 28655 ts_tablet_manager.cc:1398] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.745122 28655 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.745579 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 369e89bd1625453c8f78e99b6d40bd61 with delete_type TABLET_DATA_DELETED (replaced by tablet 138ffe8057904a2485289b61064bf382) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.745704 28655 tablet_bootstrap.cc:654] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.745848 28653 ts_tablet_manager.cc:1888] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.745893 28653 log.cc:1196] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c0d68fe045a046538ebc5b0ef87874ee
I20240503 02:09:15.746033 28653 ts_tablet_manager.cc:1909] T c0d68fe045a046538ebc5b0ef87874ee P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.746327 28653 tablet_replica.cc:330] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.746377 28653 raft_consensus.cc:2229] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.746413 28653 raft_consensus.cc:2258] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.746438 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c0d68fe045a046538ebc5b0ef87874ee (table default.Table_ffea94b33c194129ba6db6c5a2af129a [id=1b6ad2c5f6124700a404831aff1ae97d]) successfully deleted
I20240503 02:09:15.746822 28653 ts_tablet_manager.cc:1875] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.746627 28655 tablet_bootstrap.cc:492] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.746959 28655 ts_tablet_manager.cc:1367] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.747160 28655 raft_consensus.cc:348] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.747216 28655 raft_consensus.cc:374] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.747232 28655 raft_consensus.cc:729] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.747279 28655 consensus_queue.cc:260] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.747762 28655 ts_tablet_manager.cc:1398] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.747830 28655 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.747911 28653 ts_tablet_manager.cc:1888] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.747954 28653 log.cc:1196] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/369e89bd1625453c8f78e99b6d40bd61
I20240503 02:09:15.748047 28653 ts_tablet_manager.cc:1909] T 369e89bd1625453c8f78e99b6d40bd61 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.748155 28655 tablet_bootstrap.cc:654] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:15.748502 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:15.748955 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 369e89bd1625453c8f78e99b6d40bd61 (table default.Table_6dea9657852d452dbf25d95e393bbd74 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:15.749048 28655 tablet_bootstrap.cc:492] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.749086 28655 ts_tablet_manager.cc:1367] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.749286 28655 raft_consensus.cc:348] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.749361 28655 raft_consensus.cc:374] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.749440 28655 raft_consensus.cc:729] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.749480 28655 consensus_queue.cc:260] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.749720 28655 ts_tablet_manager.cc:1398] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.758647 28774 raft_consensus.cc:482] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.758699 28774 raft_consensus.cc:504] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.758821 28774 leader_election.cc:288] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.758960 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffe8057904a2485289b61064bf382" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.759018 27449 raft_consensus.cc:2454] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.759081 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffe8057904a2485289b61064bf382" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.759133 27582 raft_consensus.cc:2454] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.759254 27656 leader_election.cc:302] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.759356 28774 raft_consensus.cc:2781] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.759406 28774 raft_consensus.cc:482] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.759431 28774 raft_consensus.cc:3037] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.760003 28774 raft_consensus.cc:504] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.760123 28774 leader_election.cc:288] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.760309 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffe8057904a2485289b61064bf382" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.760361 27449 raft_consensus.cc:3037] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.760788 27449 raft_consensus.cc:2454] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.760874 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "138ffe8057904a2485289b61064bf382" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.760931 27582 raft_consensus.cc:3037] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.760938 27659 leader_election.cc:302] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.761018 28774 raft_consensus.cc:2781] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.761054 28774 raft_consensus.cc:686] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.761085 28774 consensus_queue.cc:237] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.761559 27582 raft_consensus.cc:2454] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.762239 27198 catalog_manager.cc:5484] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.768898 28740 consensus_queue.cc:1035] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.770678 28693 consensus_queue.cc:1035] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.777654 28767 raft_consensus.cc:482] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.777706 28767 raft_consensus.cc:504] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.777793 28767 leader_election.cc:288] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.777904 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab33852711d4212b1603fa04904cf0a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.777922 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab33852711d4212b1603fa04904cf0a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.777969 27719 raft_consensus.cc:2454] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.777971 27582 raft_consensus.cc:2454] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.778100 27386 leader_election.cc:302] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.778193 28767 raft_consensus.cc:2781] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.778231 28767 raft_consensus.cc:482] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.778244 28767 raft_consensus.cc:3037] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.778759 28767 raft_consensus.cc:504] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.778861 28767 leader_election.cc:288] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.778980 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab33852711d4212b1603fa04904cf0a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.778970 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ab33852711d4212b1603fa04904cf0a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.779031 27719 raft_consensus.cc:3037] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.779035 27582 raft_consensus.cc:3037] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.779237 28767 consensus_queue.cc:1035] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.779613 27582 raft_consensus.cc:2454] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.779711 27719 raft_consensus.cc:2454] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.779935 27388 leader_election.cc:302] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.780028 28767 raft_consensus.cc:2781] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.780061 28767 raft_consensus.cc:686] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.780097 28767 consensus_queue.cc:237] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.781602 27198 catalog_manager.cc:5484] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.781955 28767 consensus_queue.cc:1035] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.787010 28410 consensus_queue.cc:1035] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.788676 27848 consensus_queue.cc:1035] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.794162 28767 raft_consensus.cc:482] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.794214 28767 raft_consensus.cc:504] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.794332 28767 leader_election.cc:288] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.794483 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d7641f83dc4d029ecd170a2d14888b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.794538 27587 raft_consensus.cc:2454] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.794517 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d7641f83dc4d029ecd170a2d14888b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.794574 27719 raft_consensus.cc:2454] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.794687 27386 leader_election.cc:302] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.794790 28766 raft_consensus.cc:2781] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.794822 28766 raft_consensus.cc:482] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.794844 28766 raft_consensus.cc:3037] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.795343 28766 raft_consensus.cc:504] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.795433 28766 leader_election.cc:288] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.795557 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d7641f83dc4d029ecd170a2d14888b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.795583 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d7641f83dc4d029ecd170a2d14888b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.795627 27719 raft_consensus.cc:3037] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.795627 27587 raft_consensus.cc:3037] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.796186 27719 raft_consensus.cc:2454] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.796211 27587 raft_consensus.cc:2454] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.796350 27388 leader_election.cc:302] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.796435 28766 raft_consensus.cc:2781] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.796494 28766 raft_consensus.cc:686] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.796536 28766 consensus_queue.cc:237] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.797247 27198 catalog_manager.cc:5484] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.801472 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f011cd6264814548a989da59664aaf6b" } modify_external_catalogs: true
I20240503 02:09:15.801537 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f011cd6264814548a989da59664aaf6b" } modify_external_catalogs: true
I20240503 02:09:15.802485 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet dd6244d9ac8b40aab5f6e2a40cda8463
I20240503 02:09:15.802731 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.802732 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.802737 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet dd6244d9ac8b40aab5f6e2a40cda8463 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.802809 28653 tablet_replica.cc:330] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.802809 28661 tablet_replica.cc:330] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.802861 28653 raft_consensus.cc:2229] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.802868 28661 raft_consensus.cc:2229] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.802888 28653 raft_consensus.cc:2258] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.802927 28661 pending_rounds.cc:62] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.802944 28661 pending_rounds.cc:68] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420029219717120 op_type: NO_OP noop_request { }
I20240503 02:09:15.802978 28661 raft_consensus.cc:2866] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.803001 28661 raft_consensus.cc:2258] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.803058 28654 tablet_replica.cc:330] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.803114 28654 raft_consensus.cc:2229] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.803126 28653 ts_tablet_manager.cc:1875] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.803141 28654 raft_consensus.cc:2258] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.803365 28654 ts_tablet_manager.cc:1875] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.803404 28661 ts_tablet_manager.cc:1875] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.804380 28653 ts_tablet_manager.cc:1888] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.804432 28653 log.cc:1196] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/dd6244d9ac8b40aab5f6e2a40cda8463
I20240503 02:09:15.804459 28661 ts_tablet_manager.cc:1888] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.804497 28661 log.cc:1196] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/dd6244d9ac8b40aab5f6e2a40cda8463
I20240503 02:09:15.804591 28653 ts_tablet_manager.cc:1909] T dd6244d9ac8b40aab5f6e2a40cda8463 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.804656 28661 ts_tablet_manager.cc:1909] T dd6244d9ac8b40aab5f6e2a40cda8463 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.804661 28654 ts_tablet_manager.cc:1888] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.804698 28654 log.cc:1196] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/dd6244d9ac8b40aab5f6e2a40cda8463
I20240503 02:09:15.804834 28654 ts_tablet_manager.cc:1909] T dd6244d9ac8b40aab5f6e2a40cda8463 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.804958 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet dd6244d9ac8b40aab5f6e2a40cda8463 (table default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]) successfully deleted
I20240503 02:09:15.805012 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet dd6244d9ac8b40aab5f6e2a40cda8463 (table default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]) successfully deleted
I20240503 02:09:15.805234 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet dd6244d9ac8b40aab5f6e2a40cda8463 (table default.Table_f011cd6264814548a989da59664aaf6b [id=28af0b7cfbbf4591b6f6a4570abfe00b]) successfully deleted
I20240503 02:09:15.805256 28774 raft_consensus.cc:482] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.805308 28774 raft_consensus.cc:504] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.805442 28774 leader_election.cc:288] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.805543 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458b6fac520a4691a9de9a76c0305353" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.805615 27449 raft_consensus.cc:2454] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.805631 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458b6fac520a4691a9de9a76c0305353" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.805704 27587 raft_consensus.cc:2454] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.805761 27659 leader_election.cc:302] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.805838 28774 raft_consensus.cc:2781] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.805867 28774 raft_consensus.cc:482] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.805891 28774 raft_consensus.cc:3037] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.806314 28774 raft_consensus.cc:504] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.806428 28774 leader_election.cc:288] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.806528 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458b6fac520a4691a9de9a76c0305353" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.806550 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458b6fac520a4691a9de9a76c0305353" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.806581 27449 raft_consensus.cc:3037] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.806599 27587 raft_consensus.cc:3037] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.806995 27449 raft_consensus.cc:2454] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.807024 27587 raft_consensus.cc:2454] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.807168 27656 leader_election.cc:302] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.807240 28774 raft_consensus.cc:2781] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.807274 28774 raft_consensus.cc:686] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.807314 28774 consensus_queue.cc:237] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.807925 27198 catalog_manager.cc:5484] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.812415 28774 raft_consensus.cc:482] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.812463 28774 raft_consensus.cc:504] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.812548 28774 leader_election.cc:288] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.812676 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a07194e9874f7f9df96fc754e25779" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.812700 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a07194e9874f7f9df96fc754e25779" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.812737 27587 raft_consensus.cc:2454] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.812745 27449 raft_consensus.cc:2454] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.812866 27659 leader_election.cc:302] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.812964 28774 raft_consensus.cc:2781] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.813009 28774 raft_consensus.cc:482] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.813027 28774 raft_consensus.cc:3037] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.813478 28774 raft_consensus.cc:504] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.813585 28774 leader_election.cc:288] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.813699 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a07194e9874f7f9df96fc754e25779" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.813750 27587 raft_consensus.cc:3037] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.814175 27587 raft_consensus.cc:2454] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.814262 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a07194e9874f7f9df96fc754e25779" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.814324 27449 raft_consensus.cc:3037] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.814332 27656 leader_election.cc:302] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.814405 28774 raft_consensus.cc:2781] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.814448 28774 raft_consensus.cc:686] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.814487 28774 consensus_queue.cc:237] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.814882 27449 raft_consensus.cc:2454] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.815182 27198 catalog_manager.cc:5484] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.819366 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1c296fc27bd0411293d5fbe8313ce1cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.819482 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c296fc27bd0411293d5fbe8313ce1cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.820685 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_40eaf19db62e4a6a80b0322cc00fa0f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.820785 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40eaf19db62e4a6a80b0322cc00fa0f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.821810 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3b2192b2b34def9eef4e390822be39 (DEFAULT_TABLE table=default.Table_1c296fc27bd0411293d5fbe8313ce1cf [id=9865564162444261a15645a92010641e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.821843 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3b2192b2b34def9eef4e390822be39 (DEFAULT_TABLE table=default.Table_1c296fc27bd0411293d5fbe8313ce1cf [id=9865564162444261a15645a92010641e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.821919 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3b2192b2b34def9eef4e390822be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.821942 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3b2192b2b34def9eef4e390822be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.822816 28664 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.822908 27848 raft_consensus.cc:482] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.822953 27848 raft_consensus.cc:504] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.823058 27848 leader_election.cc:288] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.823189 28664 tablet_bootstrap.cc:654] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.823205 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09426f1e3db54724a043f24c40f0e9f1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.823254 27719 raft_consensus.cc:2454] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.823374 27523 leader_election.cc:302] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.823446 27848 raft_consensus.cc:2781] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.823427 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09426f1e3db54724a043f24c40f0e9f1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.823482 27449 raft_consensus.cc:2454] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.823472 28656 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.823472 27848 raft_consensus.cc:482] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.823535 27848 raft_consensus.cc:3037] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.823772 28656 tablet_bootstrap.cc:654] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.824018 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3b2192b2b34def9eef4e390822be39 (DEFAULT_TABLE table=default.Table_1c296fc27bd0411293d5fbe8313ce1cf [id=9865564162444261a15645a92010641e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.824108 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3b2192b2b34def9eef4e390822be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.824133 27848 raft_consensus.cc:504] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.824234 27848 leader_election.cc:288] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.824254 28656 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.824286 28656 ts_tablet_manager.cc:1367] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.824393 28656 raft_consensus.cc:348] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.824407 27554 tablet_service.cc:1467] Processing CreateTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (DEFAULT_TABLE table=default.Table_40eaf19db62e4a6a80b0322cc00fa0f3 [id=0805af74bdea4546b84d729e82bcfec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.824434 28656 raft_consensus.cc:374] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.824452 28656 raft_consensus.cc:729] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.824477 28656 consensus_queue.cc:260] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.824492 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdab84ce3fd14cf3a7fdb838cfe9e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.824563 28656 ts_tablet_manager.cc:1398] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.824645 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09426f1e3db54724a043f24c40f0e9f1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.824692 27719 raft_consensus.cc:3037] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.825232 28655 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.825280 27719 raft_consensus.cc:2454] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.825402 27419 tablet_service.cc:1467] Processing CreateTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (DEFAULT_TABLE table=default.Table_40eaf19db62e4a6a80b0322cc00fa0f3 [id=0805af74bdea4546b84d729e82bcfec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.825485 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdab84ce3fd14cf3a7fdb838cfe9e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.825655 28655 tablet_bootstrap.cc:654] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.826090 27523 leader_election.cc:302] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.826169 28507 raft_consensus.cc:2781] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.826179 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0a52c142ef664c02989e9bfa34f3d2f1" } modify_external_catalogs: true
I20240503 02:09:15.826220 28507 raft_consensus.cc:686] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.826236 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0a52c142ef664c02989e9bfa34f3d2f1" } modify_external_catalogs: true
I20240503 02:09:15.826253 28507 consensus_queue.cc:237] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.826419 28664 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.826457 28664 ts_tablet_manager.cc:1367] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:15.826496 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09426f1e3db54724a043f24c40f0e9f1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.826543 27449 raft_consensus.cc:3037] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.826587 28664 raft_consensus.cc:348] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.826633 28664 raft_consensus.cc:374] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.826649 28664 raft_consensus.cc:729] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.826682 28664 consensus_queue.cc:260] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.826758 28664 ts_tablet_manager.cc:1398] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.827069 27449 raft_consensus.cc:2454] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.827085 28655 tablet_bootstrap.cc:492] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.827127 28655 ts_tablet_manager.cc:1367] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.827258 28655 raft_consensus.cc:348] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.827316 28655 raft_consensus.cc:374] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.827333 28655 raft_consensus.cc:729] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.827368 28655 consensus_queue.cc:260] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.827428 27691 tablet_service.cc:1467] Processing CreateTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (DEFAULT_TABLE table=default.Table_40eaf19db62e4a6a80b0322cc00fa0f3 [id=0805af74bdea4546b84d729e82bcfec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.827486 28655 ts_tablet_manager.cc:1398] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.827538 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdab84ce3fd14cf3a7fdb838cfe9e648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.827584 28655 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.828016 28655 tablet_bootstrap.cc:654] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.828454 28664 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.828804 28664 tablet_bootstrap.cc:654] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.829095 27197 catalog_manager.cc:5484] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.829190 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.829300 28656 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.829449 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 458b6fac520a4691a9de9a76c0305353
I20240503 02:09:15.829588 28656 tablet_bootstrap.cc:654] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.829743 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 458b6fac520a4691a9de9a76c0305353 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.829882 28654 tablet_replica.cc:330] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.829931 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 458b6fac520a4691a9de9a76c0305353 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.829958 28654 raft_consensus.cc:2229] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.829991 28654 raft_consensus.cc:2258] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.829995 28661 tablet_replica.cc:330] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.830041 28661 raft_consensus.cc:2229] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.830080 28661 pending_rounds.cc:62] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:15.830101 28661 pending_rounds.cc:68] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420030186967040 op_type: NO_OP noop_request { }
I20240503 02:09:15.830130 28661 raft_consensus.cc:2866] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.830150 28661 raft_consensus.cc:2258] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.830106 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_69373c98e90d49f787b21be369651a47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.830193 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69373c98e90d49f787b21be369651a47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.830333 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet 458b6fac520a4691a9de9a76c0305353 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.830366 28654 ts_tablet_manager.cc:1875] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.830605 28661 ts_tablet_manager.cc:1875] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.830632 27194 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_4bda2dc4192e4d2c87ff89620b509cd7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.830724 27194 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4bda2dc4192e4d2c87ff89620b509cd7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.831681 28655 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.831718 28655 ts_tablet_manager.cc:1367] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.831835 28655 raft_consensus.cc:348] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.831887 28655 raft_consensus.cc:374] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.831903 28655 raft_consensus.cc:729] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.831879 28654 ts_tablet_manager.cc:1888] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.831952 28654 log.cc:1196] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/458b6fac520a4691a9de9a76c0305353
I20240503 02:09:15.831940 28655 consensus_queue.cc:260] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.832036 28655 ts_tablet_manager.cc:1398] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.832114 28654 ts_tablet_manager.cc:1909] T 458b6fac520a4691a9de9a76c0305353 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.832379 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 458b6fac520a4691a9de9a76c0305353 (table default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]) successfully deleted
I20240503 02:09:15.832989 28661 ts_tablet_manager.cc:1888] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.833036 28661 log.cc:1196] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/458b6fac520a4691a9de9a76c0305353
I20240503 02:09:15.833207 28661 ts_tablet_manager.cc:1909] T 458b6fac520a4691a9de9a76c0305353 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.833500 27848 raft_consensus.cc:482] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.833524 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 458b6fac520a4691a9de9a76c0305353 (table default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]) successfully deleted
I20240503 02:09:15.833549 27848 raft_consensus.cc:504] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.833658 27848 leader_election.cc:288] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.833837 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76ce5e466e64b5cae709aecaa1f4c0e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.833904 27719 raft_consensus.cc:2454] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.833946 28656 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.833981 28656 ts_tablet_manager.cc:1367] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.834066 28656 raft_consensus.cc:348] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.834120 28656 raft_consensus.cc:374] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.834141 28656 raft_consensus.cc:729] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.834175 28656 consensus_queue.cc:260] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.834259 27523 leader_election.cc:302] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.834301 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76ce5e466e64b5cae709aecaa1f4c0e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.834350 27449 raft_consensus.cc:2454] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.834379 28410 raft_consensus.cc:2781] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.834259 28656 ts_tablet_manager.cc:1398] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.834405 28410 raft_consensus.cc:482] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.834595 28410 raft_consensus.cc:3037] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.834995 28664 tablet_bootstrap.cc:492] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.835023 28664 ts_tablet_manager.cc:1367] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:15.835120 28664 raft_consensus.cc:348] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.835155 28664 raft_consensus.cc:374] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.835165 28664 raft_consensus.cc:729] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.835136 28410 raft_consensus.cc:504] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.835197 28664 consensus_queue.cc:260] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.835230 28410 leader_election.cc:288] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.835284 28664 ts_tablet_manager.cc:1398] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.835574 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76ce5e466e64b5cae709aecaa1f4c0e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.835633 27719 raft_consensus.cc:3037] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.836169 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 811e1aa52cb2458792359e9baebd5e5c (DEFAULT_TABLE table=default.Table_4bda2dc4192e4d2c87ff89620b509cd7 [id=076a99c72f8645e59fbc62a96e3887b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.836176 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb (DEFAULT_TABLE table=default.Table_69373c98e90d49f787b21be369651a47 [id=db75dd5ce72f4d18bf7363df6ff19274]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.836259 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811e1aa52cb2458792359e9baebd5e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.836263 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eb4d0cbce545c8ad87883b9be0f6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.837111 28664 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.837193 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb (DEFAULT_TABLE table=default.Table_69373c98e90d49f787b21be369651a47 [id=db75dd5ce72f4d18bf7363df6ff19274]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.837273 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eb4d0cbce545c8ad87883b9be0f6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.837280 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 811e1aa52cb2458792359e9baebd5e5c (DEFAULT_TABLE table=default.Table_4bda2dc4192e4d2c87ff89620b509cd7 [id=076a99c72f8645e59fbc62a96e3887b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.837388 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811e1aa52cb2458792359e9baebd5e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.837539 28664 tablet_bootstrap.cc:654] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.837630 28653 tablet_replica.cc:330] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.837685 28653 raft_consensus.cc:2229] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.837715 28653 raft_consensus.cc:2258] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.838032 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 811e1aa52cb2458792359e9baebd5e5c (DEFAULT_TABLE table=default.Table_4bda2dc4192e4d2c87ff89620b509cd7 [id=076a99c72f8645e59fbc62a96e3887b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.838114 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811e1aa52cb2458792359e9baebd5e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.838361 28653 ts_tablet_manager.cc:1875] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.838754 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76ce5e466e64b5cae709aecaa1f4c0e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.838809 27449 raft_consensus.cc:3037] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.839164 28664 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.839200 28664 ts_tablet_manager.cc:1367] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.839287 27449 raft_consensus.cc:2454] T c76ce5e466e64b5cae709aecaa1f4c0e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.839346 28656 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.839334 28664 raft_consensus.cc:348] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.839385 28664 raft_consensus.cc:374] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.839401 28664 raft_consensus.cc:729] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.839437 28664 consensus_queue.cc:260] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.839546 28664 ts_tablet_manager.cc:1398] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.839601 28664 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.839612 28656 tablet_bootstrap.cc:654] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.839651 28653 ts_tablet_manager.cc:1888] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.839708 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb (DEFAULT_TABLE table=default.Table_69373c98e90d49f787b21be369651a47 [id=db75dd5ce72f4d18bf7363df6ff19274]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.839800 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62eb4d0cbce545c8ad87883b9be0f6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.839929 28664 tablet_bootstrap.cc:654] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.840102 27524 leader_election.cc:302] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.840217 28655 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.840260 28410 raft_consensus.cc:2781] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.840298 28410 raft_consensus.cc:686] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.840333 28410 consensus_queue.cc:237] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.840584 28655 tablet_bootstrap.cc:654] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.836174 27719 raft_consensus.cc:2454] T c76ce5e466e64b5cae709aecaa1f4c0e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.841348 27194 catalog_manager.cc:5484] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.841894 28655 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.841936 28655 ts_tablet_manager.cc:1367] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:15.842060 28655 raft_consensus.cc:348] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.842116 28655 raft_consensus.cc:374] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.842154 28655 raft_consensus.cc:729] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.842185 28655 consensus_queue.cc:260] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.842278 28655 ts_tablet_manager.cc:1398] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.842425 28655 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.842777 28655 tablet_bootstrap.cc:654] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.842876 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f5965cead947400690406a4c61d35b7e" } modify_external_catalogs: true
I20240503 02:09:15.842937 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f5965cead947400690406a4c61d35b7e" } modify_external_catalogs: true
I20240503 02:09:15.843588 28655 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.843631 28655 ts_tablet_manager.cc:1367] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.839700 28653 log.cc:1196] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/458b6fac520a4691a9de9a76c0305353
I20240503 02:09:15.843773 28655 raft_consensus.cc:348] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.843823 28655 raft_consensus.cc:374] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.843838 28655 raft_consensus.cc:729] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.843869 28655 consensus_queue.cc:260] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.843922 28653 ts_tablet_manager.cc:1909] T 458b6fac520a4691a9de9a76c0305353 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.843959 28655 ts_tablet_manager.cc:1398] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.844594 27194 master_service.cc:946] ReplaceTablet: received request to replace tablet f7c9c3fa8cd1495bb93818b2879ecfd4 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.845265 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ec48a9db1d1d4c129f07e4e0eb9918ca
I20240503 02:09:15.845530 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.845598 28661 tablet_replica.cc:330] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.845566 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f645395dbcd54876beef5a179ddf4814"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.845640 28661 raft_consensus.cc:2229] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.845662 28661 raft_consensus.cc:2258] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:15.845659 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f645395dbcd54876beef5a179ddf4814 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.845796 28661 ts_tablet_manager.cc:1875] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.845785 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bc72842daac14d2cbe4e8d4b52418c8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.845856 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc72842daac14d2cbe4e8d4b52418c8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.845958 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.846062 28654 tablet_replica.cc:330] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.846132 28654 raft_consensus.cc:2229] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.846174 28654 pending_rounds.cc:62] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:15.846189 28654 pending_rounds.cc:68] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420029107511296 op_type: NO_OP noop_request { }
I20240503 02:09:15.846220 28654 raft_consensus.cc:2866] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.846240 28654 raft_consensus.cc:2258] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.846362 28664 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.846402 28664 ts_tablet_manager.cc:1367] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:15.846522 28664 raft_consensus.cc:348] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.846509 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3237c35b4a7d4fffb9e90e8584316e72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.846568 28664 raft_consensus.cc:374] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.846583 28664 raft_consensus.cc:729] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
W20240503 02:09:15.846589 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3237c35b4a7d4fffb9e90e8584316e72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.846621 28664 consensus_queue.cc:260] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.846704 28664 ts_tablet_manager.cc:1398] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.846777 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_32834d5c492a480090b2c41a0497a60f" } new_table_name: "default.Table_34df9f9639b74f71afa144d062232515" modify_external_catalogs: true
I20240503 02:09:15.846840 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_c173d5f4572049b88bbef92d8090e135"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:15.846906 28661 ts_tablet_manager.cc:1888] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20240503 02:09:15.846922 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c173d5f4572049b88bbef92d8090e135 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.846946 28661 log.cc:1196] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ec48a9db1d1d4c129f07e4e0eb9918ca
I20240503 02:09:15.847086 28661 ts_tablet_manager.cc:1909] T ec48a9db1d1d4c129f07e4e0eb9918ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.847508 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ec48a9db1d1d4c129f07e4e0eb9918ca (table default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]) successfully deleted
I20240503 02:09:15.847841 28654 ts_tablet_manager.cc:1875] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.847887 27194 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f7c9c3fa8cd1495bb93818b2879ecfd4
I20240503 02:09:15.847967 27194 catalog_manager.cc:6554] ReplaceTablet: tablet f7c9c3fa8cd1495bb93818b2879ecfd4 deleted and replaced by tablet 06be437ed5114166b802559170e440eb
I20240503 02:09:15.848284 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 06be437ed5114166b802559170e440eb) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.848634 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ef979a5f65c4465987fed4ac2de01fb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.848719 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef979a5f65c4465987fed4ac2de01fb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.848989 28654 ts_tablet_manager.cc:1888] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.849014 28656 tablet_bootstrap.cc:492] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.849030 28654 log.cc:1196] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ec48a9db1d1d4c129f07e4e0eb9918ca
I20240503 02:09:15.849045 28656 ts_tablet_manager.cc:1367] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240503 02:09:15.849131 28656 raft_consensus.cc:348] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.849176 28656 raft_consensus.cc:374] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.849193 28656 raft_consensus.cc:729] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.849205 28654 ts_tablet_manager.cc:1909] T ec48a9db1d1d4c129f07e4e0eb9918ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.849231 28656 consensus_queue.cc:260] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.849396 28656 ts_tablet_manager.cc:1398] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.849452 28656 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.849663 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 06be437ed5114166b802559170e440eb) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.849808 28656 tablet_bootstrap.cc:654] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.849812 28661 tablet_replica.cc:330] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.849829 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet ec48a9db1d1d4c129f07e4e0eb9918ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.849871 28661 raft_consensus.cc:2229] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.849892 28653 tablet_replica.cc:330] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.849903 28661 raft_consensus.cc:2258] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.849939 28653 raft_consensus.cc:2229] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.849967 28653 raft_consensus.cc:2258] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.850241 28653 ts_tablet_manager.cc:1875] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.850323 28661 ts_tablet_manager.cc:1875] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.850394 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 06be437ed5114166b802559170e440eb (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.850466 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06be437ed5114166b802559170e440eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.851523 28661 ts_tablet_manager.cc:1888] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.851538 28653 ts_tablet_manager.cc:1888] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.851565 28661 log.cc:1196] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f7c9c3fa8cd1495bb93818b2879ecfd4
I20240503 02:09:15.851580 28653 log.cc:1196] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ec48a9db1d1d4c129f07e4e0eb9918ca
I20240503 02:09:15.851748 28653 ts_tablet_manager.cc:1909] T ec48a9db1d1d4c129f07e4e0eb9918ca P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.851750 28661 ts_tablet_manager.cc:1909] T f7c9c3fa8cd1495bb93818b2879ecfd4 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.852007 29172 tablet.cc:1722] T 11a234271dc14a04a6feeb30e2fcd8f2 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.852025 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.852173 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 06be437ed5114166b802559170e440eb (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.852250 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06be437ed5114166b802559170e440eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.853055 28664 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.853364 28710 tablet.cc:1722] T 11a234271dc14a04a6feeb30e2fcd8f2 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.853583 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 458b6fac520a4691a9de9a76c0305353 (table default.Table_0a52c142ef664c02989e9bfa34f3d2f1 [id=d51b1f85867c4877b4d9e20db283cff6]) successfully deleted
I20240503 02:09:15.853744 28664 tablet_bootstrap.cc:654] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.853933 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet f7c9c3fa8cd1495bb93818b2879ecfd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 06be437ed5114166b802559170e440eb) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.853955 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ec48a9db1d1d4c129f07e4e0eb9918ca (table default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]) successfully deleted
I20240503 02:09:15.854015 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 06be437ed5114166b802559170e440eb (DEFAULT_TABLE table=default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.854164 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06be437ed5114166b802559170e440eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.855569 28653 tablet_replica.cc:330] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.855631 28653 raft_consensus.cc:2229] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.855684 28653 raft_consensus.cc:2258] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.855746 28655 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.855885 28653 ts_tablet_manager.cc:1875] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.856060 28655 tablet_bootstrap.cc:654] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.856344 28698 tablet.cc:1722] T 11a234271dc14a04a6feeb30e2fcd8f2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.856523 27848 consensus_queue.cc:1035] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.857038 28653 ts_tablet_manager.cc:1888] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.857080 28653 log.cc:1196] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f7c9c3fa8cd1495bb93818b2879ecfd4
I20240503 02:09:15.857126 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 34c4d03198554e4aa7a11125f851debc (DEFAULT_TABLE table=default.Table_c173d5f4572049b88bbef92d8090e135 [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.857148 27693 tablet_service.cc:1467] Processing CreateTablet for tablet fa960c0281414895a12972c4c1e5269c (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.857208 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa960c0281414895a12972c4c1e5269c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.857193 27694 tablet_service.cc:1467] Processing CreateTablet for tablet a8ce961ecae44ca18642f64cf3f02977 (DEFAULT_TABLE table=default.Table_3237c35b4a7d4fffb9e90e8584316e72 [id=a7f45eb95c1e41458f2ecd297f2663e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.857363 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ce961ecae44ca18642f64cf3f02977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.857470 27554 tablet_service.cc:1467] Processing CreateTablet for tablet ce779dcce25548eba2fc39908daf2b9a (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.857556 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce779dcce25548eba2fc39908daf2b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.857620 28664 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.857653 28664 ts_tablet_manager.cc:1367] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:15.857764 28664 raft_consensus.cc:348] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.857810 28664 raft_consensus.cc:374] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.857826 28664 raft_consensus.cc:729] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.857859 28664 consensus_queue.cc:260] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.857944 28664 ts_tablet_manager.cc:1398] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.858325 27703 tablet_service.cc:1467] Processing CreateTablet for tablet fd9df0eb41d54a7ca550c0323c305424 (DEFAULT_TABLE table=default.Table_ef979a5f65c4465987fed4ac2de01fb8 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.858345 27562 tablet_service.cc:1467] Processing CreateTablet for tablet fa960c0281414895a12972c4c1e5269c (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.858409 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9df0eb41d54a7ca550c0323c305424. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.858413 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa960c0281414895a12972c4c1e5269c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.858440 28664 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.858546 27557 tablet_service.cc:1467] Processing CreateTablet for tablet fd9df0eb41d54a7ca550c0323c305424 (DEFAULT_TABLE table=default.Table_ef979a5f65c4465987fed4ac2de01fb8 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.858628 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9df0eb41d54a7ca550c0323c305424. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.858832 28664 tablet_bootstrap.cc:654] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.859426 27560 tablet_service.cc:1467] Processing CreateTablet for tablet a8ce961ecae44ca18642f64cf3f02977 (DEFAULT_TABLE table=default.Table_3237c35b4a7d4fffb9e90e8584316e72 [id=a7f45eb95c1e41458f2ecd297f2663e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.859440 27692 tablet_service.cc:1467] Processing CreateTablet for tablet ce779dcce25548eba2fc39908daf2b9a (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.859508 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce779dcce25548eba2fc39908daf2b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.859508 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ce961ecae44ca18642f64cf3f02977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.859645 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_60fd4282c8764278a417ee4f30f5686b" } new_table_name: "default.Table_97f6a0dc82a840dd82a2fc7b3591801d" modify_external_catalogs: true
I20240503 02:09:15.860220 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 34c4d03198554e4aa7a11125f851debc (DEFAULT_TABLE table=default.Table_c173d5f4572049b88bbef92d8090e135 [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.860291 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c4d03198554e4aa7a11125f851debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.860608 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c4d03198554e4aa7a11125f851debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.861466 27449 raft_consensus.cc:3037] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:15.861869 27200 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d4a1909677544549029137b79501235 consensus state has no leader
I20240503 02:09:15.862020 28653 ts_tablet_manager.cc:1909] T f7c9c3fa8cd1495bb93818b2879ecfd4 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.862166 28774 consensus_queue.cc:1035] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.862293 27419 tablet_service.cc:1467] Processing CreateTablet for tablet ce779dcce25548eba2fc39908daf2b9a (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.862320 27425 tablet_service.cc:1467] Processing CreateTablet for tablet fa960c0281414895a12972c4c1e5269c (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.862373 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce779dcce25548eba2fc39908daf2b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.862375 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.862483 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa960c0281414895a12972c4c1e5269c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.862793 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_34df9f9639b74f71afa144d062232515 [id=cb1132c7b84948a6b0438c3f0bf5a53d] alter complete (version 1)
I20240503 02:09:15.863448 27426 tablet_service.cc:1467] Processing CreateTablet for tablet fd9df0eb41d54a7ca550c0323c305424 (DEFAULT_TABLE table=default.Table_ef979a5f65c4465987fed4ac2de01fb8 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.863512 28654 tablet_replica.cc:330] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.863533 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd9df0eb41d54a7ca550c0323c305424. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.863567 28654 raft_consensus.cc:2229] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.863597 28654 raft_consensus.cc:2258] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.863772 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ec48a9db1d1d4c129f07e4e0eb9918ca (table default.Table_f5965cead947400690406a4c61d35b7e [id=1ec574a59320464599c773476c483b53]) successfully deleted
I20240503 02:09:15.864282 28654 ts_tablet_manager.cc:1875] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.864553 27433 tablet_service.cc:1467] Processing CreateTablet for tablet a8ce961ecae44ca18642f64cf3f02977 (DEFAULT_TABLE table=default.Table_3237c35b4a7d4fffb9e90e8584316e72 [id=a7f45eb95c1e41458f2ecd297f2663e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.864626 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ce961ecae44ca18642f64cf3f02977. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.865406 28654 ts_tablet_manager.cc:1888] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.865447 28654 log.cc:1196] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f7c9c3fa8cd1495bb93818b2879ecfd4
I20240503 02:09:15.865463 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ef7ad145af794a2fab93caf50bce1418" } new_table_name: "default.Table_676914005e65475eac45ad325ccae621" modify_external_catalogs: true
I20240503 02:09:15.865613 28654 ts_tablet_manager.cc:1909] T f7c9c3fa8cd1495bb93818b2879ecfd4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.865742 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a273104fb204435d9b452271f100bb80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:15.865828 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a273104fb204435d9b452271f100bb80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:15.866204 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f7c9c3fa8cd1495bb93818b2879ecfd4 (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:15.867161 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 34c4d03198554e4aa7a11125f851debc (DEFAULT_TABLE table=default.Table_c173d5f4572049b88bbef92d8090e135 [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.867242 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34c4d03198554e4aa7a11125f851debc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.869438 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_185b871b5e1049c28c3690a576682aaa" } new_table_name: "default.Table_709fdd1f272c489a815f79b02bfa18b3" modify_external_catalogs: true
I20240503 02:09:15.869658 27848 raft_consensus.cc:482] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.869699 27848 raft_consensus.cc:504] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.869736 27449 raft_consensus.cc:1261] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.869781 27848 leader_election.cc:288] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.870227 28774 consensus_queue.cc:1035] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.870402 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3b2192b2b34def9eef4e390822be39" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.870447 27449 raft_consensus.cc:2454] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.870568 27848 consensus_queue.cc:1035] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.871307 28766 raft_consensus.cc:482] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.871351 28766 raft_consensus.cc:504] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.871438 28766 leader_election.cc:288] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.871565 27524 leader_election.cc:302] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.871670 28410 raft_consensus.cc:2781] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.871699 28410 raft_consensus.cc:482] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.871716 28410 raft_consensus.cc:3037] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.872056 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdab84ce3fd14cf3a7fdb838cfe9e648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.872113 27587 raft_consensus.cc:2454] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.872324 28410 raft_consensus.cc:504] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.872411 28410 leader_election.cc:288] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:15.872460 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.872560 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdab84ce3fd14cf3a7fdb838cfe9e648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.872617 27723 raft_consensus.cc:2454] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.873040 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 47a4f79dd7ac4d929abda2d67aad87d9 (DEFAULT_TABLE table=default.Table_a273104fb204435d9b452271f100bb80 [id=b201298613b94cd79c63ab8832cde060]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.873049 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 47a4f79dd7ac4d929abda2d67aad87d9 (DEFAULT_TABLE table=default.Table_a273104fb204435d9b452271f100bb80 [id=b201298613b94cd79c63ab8832cde060]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.873066 27388 leader_election.cc:302] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.873128 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a4f79dd7ac4d929abda2d67aad87d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.873140 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a4f79dd7ac4d929abda2d67aad87d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.873150 28766 raft_consensus.cc:2781] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.873183 28766 raft_consensus.cc:482] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.873201 28766 raft_consensus.cc:3037] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.873816 28766 raft_consensus.cc:504] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.873917 28766 leader_election.cc:288] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.873925 27720 raft_consensus.cc:1261] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.874027 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3b2192b2b34def9eef4e390822be39" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.874083 27722 raft_consensus.cc:2454] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.874172 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3b2192b2b34def9eef4e390822be39" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.874208 27718 raft_consensus.cc:3037] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.874549 27848 consensus_queue.cc:1035] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.874783 27718 raft_consensus.cc:2454] T 2e3b2192b2b34def9eef4e390822be39 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.874989 27723 raft_consensus.cc:1261] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.875779 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 42dc7b64b7d4484296de6b8fc78e2705 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.875819 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2b675ecf68b049edac996e4fc7b0bbd8 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:15.875902 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3b2192b2b34def9eef4e390822be39" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.875953 27450 raft_consensus.cc:3037] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.876003 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 47a4f79dd7ac4d929abda2d67aad87d9 (DEFAULT_TABLE table=default.Table_a273104fb204435d9b452271f100bb80 [id=b201298613b94cd79c63ab8832cde060]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.876085 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a4f79dd7ac4d929abda2d67aad87d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.876374 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet d3152be657da44e3ae7fd9501842c6ac from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.876559 27450 raft_consensus.cc:2454] T 2e3b2192b2b34def9eef4e390822be39 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.876644 27453 raft_consensus.cc:1261] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.876691 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdab84ce3fd14cf3a7fdb838cfe9e648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.876761 27722 raft_consensus.cc:3037] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.876976 27523 leader_election.cc:302] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.877115 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdab84ce3fd14cf3a7fdb838cfe9e648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.877157 28507 raft_consensus.cc:2781] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.877218 27848 consensus_queue.cc:1035] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.877245 28507 raft_consensus.cc:686] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.877285 28507 consensus_queue.cc:237] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.877353 27722 raft_consensus.cc:2454] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.877444 28698 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.877171 27587 raft_consensus.cc:3037] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.877807 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 42dc7b64b7d4484296de6b8fc78e2705
I20240503 02:09:15.877882 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 42dc7b64b7d4484296de6b8fc78e2705 deleted and replaced by tablet 78e562fa324448f788adaa4f60da3076
I20240503 02:09:15.878166 27587 raft_consensus.cc:2454] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.878265 27388 leader_election.cc:302] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.878263 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e562fa324448f788adaa4f60da3076) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.878340 28654 tablet_replica.cc:330] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.878396 28654 raft_consensus.cc:2229] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.878428 28654 raft_consensus.cc:2258] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.878504 28654 ts_tablet_manager.cc:1875] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.878646 28767 raft_consensus.cc:2781] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.878695 28767 raft_consensus.cc:686] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.878728 28767 consensus_queue.cc:237] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.878775 28710 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.878865 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e562fa324448f788adaa4f60da3076) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.878939 28653 tablet_replica.cc:330] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.878993 28653 raft_consensus.cc:2229] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.879040 28653 raft_consensus.cc:2258] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.879124 28653 ts_tablet_manager.cc:1875] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.879463 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bd6940d56b554a6096a02a7ac3fb991f" } modify_external_catalogs: true
I20240503 02:09:15.879521 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bd6940d56b554a6096a02a7ac3fb991f" } modify_external_catalogs: true
I20240503 02:09:15.879575 28654 ts_tablet_manager.cc:1888] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.879607 28654 log.cc:1196] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/42dc7b64b7d4484296de6b8fc78e2705
I20240503 02:09:15.879762 28654 ts_tablet_manager.cc:1909] T 42dc7b64b7d4484296de6b8fc78e2705 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.880169 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 42dc7b64b7d4484296de6b8fc78e2705 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:15.880264 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet c993bef72a984db794e4cad2aca7af72 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.880306 28653 ts_tablet_manager.cc:1888] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.880298 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a58c505992a54fdfa6be0c230dd83adb" } new_table_name: "default.Table_87eb7af059c842d6b7b4c8e2be015384" modify_external_catalogs: true
I20240503 02:09:15.880297 28410 consensus_queue.cc:1035] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.880343 28653 log.cc:1196] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/42dc7b64b7d4484296de6b8fc78e2705
I20240503 02:09:15.880543 28653 ts_tablet_manager.cc:1909] T 42dc7b64b7d4484296de6b8fc78e2705 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.880722 27194 catalog_manager.cc:5484] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.880810 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d3152be657da44e3ae7fd9501842c6ac
I20240503 02:09:15.880892 27196 catalog_manager.cc:6554] ReplaceTablet: tablet d3152be657da44e3ae7fd9501842c6ac deleted and replaced by tablet 9b66e153765c42128c4f64f0894fbe87
I20240503 02:09:15.881417 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet d3152be657da44e3ae7fd9501842c6ac with delete_type TABLET_DATA_DELETED (replaced by tablet 9b66e153765c42128c4f64f0894fbe87) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.882282 27722 raft_consensus.cc:1261] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.882493 28661 tablet_replica.cc:330] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.882548 28661 raft_consensus.cc:2229] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.882579 28661 raft_consensus.cc:2258] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.883210 27449 raft_consensus.cc:1261] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.883219 28410 consensus_queue.cc:1035] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.883419 29241 raft_consensus.cc:482] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.883466 29241 raft_consensus.cc:504] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.883636 29241 leader_election.cc:288] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.883672 29241 consensus_queue.cc:1035] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.883867 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3630cebdb1b34cebab8118c250543ade" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.883914 27449 raft_consensus.cc:2454] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.884088 27524 leader_election.cc:302] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.884193 28507 raft_consensus.cc:2781] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.884188 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3630cebdb1b34cebab8118c250543ade" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.884220 28507 raft_consensus.cc:482] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.884238 28507 raft_consensus.cc:3037] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.884240 27723 raft_consensus.cc:2454] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.884687 28698 tablet.cc:1722] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.884768 28507 raft_consensus.cc:504] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.884852 28507 leader_election.cc:288] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.885092 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3630cebdb1b34cebab8118c250543ade" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.885143 27449 raft_consensus.cc:3037] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.885707 27449 raft_consensus.cc:2454] T 3630cebdb1b34cebab8118c250543ade P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.885847 27524 leader_election.cc:302] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.885915 28507 raft_consensus.cc:2781] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.885946 28507 raft_consensus.cc:686] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.885977 28507 consensus_queue.cc:237] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.885995 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3630cebdb1b34cebab8118c250543ade" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.886045 27723 raft_consensus.cc:3037] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.886597 27723 raft_consensus.cc:2454] T 3630cebdb1b34cebab8118c250543ade P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.886751 29172 tablet.cc:1722] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.886844 28686 tablet.cc:1722] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.887151 28706 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.889600 28507 consensus_queue.cc:1035] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.891480 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 42dc7b64b7d4484296de6b8fc78e2705 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e562fa324448f788adaa4f60da3076) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.891600 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet d3152be657da44e3ae7fd9501842c6ac with delete_type TABLET_DATA_DELETED (replaced by tablet 9b66e153765c42128c4f64f0894fbe87) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.891672 28653 tablet_replica.cc:330] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.891727 28653 raft_consensus.cc:2229] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.891757 28653 raft_consensus.cc:2258] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.892030 28653 ts_tablet_manager.cc:1875] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.892995 28653 ts_tablet_manager.cc:1888] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.893050 28653 log.cc:1196] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d3152be657da44e3ae7fd9501842c6ac
I20240503 02:09:15.893224 28653 ts_tablet_manager.cc:1909] T d3152be657da44e3ae7fd9501842c6ac P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.893750 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet d3152be657da44e3ae7fd9501842c6ac with delete_type TABLET_DATA_DELETED (replaced by tablet 9b66e153765c42128c4f64f0894fbe87) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.893831 28698 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.893898 28654 tablet_replica.cc:330] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.893950 28654 raft_consensus.cc:2229] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.894004 28654 raft_consensus.cc:2258] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.894308 28654 ts_tablet_manager.cc:1875] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.894819 28661 ts_tablet_manager.cc:1875] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.895421 28654 ts_tablet_manager.cc:1888] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.895463 28654 log.cc:1196] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d3152be657da44e3ae7fd9501842c6ac
I20240503 02:09:15.895478 28656 tablet_bootstrap.cc:492] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.895527 28656 ts_tablet_manager.cc:1367] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.001s
I20240503 02:09:15.895624 28654 ts_tablet_manager.cc:1909] T d3152be657da44e3ae7fd9501842c6ac P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.895642 28656 raft_consensus.cc:348] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.895690 28656 raft_consensus.cc:374] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.895706 28656 raft_consensus.cc:729] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.895740 28656 consensus_queue.cc:260] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.895817 28656 ts_tablet_manager.cc:1398] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.895859 28656 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.895994 28661 ts_tablet_manager.cc:1888] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.896039 28661 log.cc:1196] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d3152be657da44e3ae7fd9501842c6ac
I20240503 02:09:15.896138 28710 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.896152 28656 tablet_bootstrap.cc:654] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.896209 28661 ts_tablet_manager.cc:1909] T d3152be657da44e3ae7fd9501842c6ac P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.896536 28661 tablet_replica.cc:330] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.896592 28661 raft_consensus.cc:2229] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.896617 28661 raft_consensus.cc:2258] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.896710 28661 ts_tablet_manager.cc:1875] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.881561 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f8419275f80940f4aad5ebc259cb52fc
I20240503 02:09:15.896749 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d3152be657da44e3ae7fd9501842c6ac (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:15.882387 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 42dc7b64b7d4484296de6b8fc78e2705 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:15.896924 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d3152be657da44e3ae7fd9501842c6ac (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:15.882933 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5ee8aa23d34f421f8f05b0982609667a" } modify_external_catalogs: true
I20240503 02:09:15.897390 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5ee8aa23d34f421f8f05b0982609667a" } modify_external_catalogs: true
I20240503 02:09:15.897432 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet f8419275f80940f4aad5ebc259cb52fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.890589 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c993bef72a984db794e4cad2aca7af72
I20240503 02:09:15.897620 27198 catalog_manager.cc:6554] ReplaceTablet: tablet c993bef72a984db794e4cad2aca7af72 deleted and replaced by tablet 4782d97b11e04c0ba788b9af68ab8802
I20240503 02:09:15.890625 27201 catalog_manager.cc:5484] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.897691 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet f8419275f80940f4aad5ebc259cb52fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.897948 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet c993bef72a984db794e4cad2aca7af72 with delete_type TABLET_DATA_DELETED (replaced by tablet 4782d97b11e04c0ba788b9af68ab8802) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.893808 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 80e4ce547683468a9cb0e1683b248508 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:15.896453 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d3152be657da44e3ae7fd9501842c6ac (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:15.898294 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet c993bef72a984db794e4cad2aca7af72 with delete_type TABLET_DATA_DELETED (replaced by tablet 4782d97b11e04c0ba788b9af68ab8802) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.898362 28655 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.898402 28655 ts_tablet_manager.cc:1367] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.000s
I20240503 02:09:15.898506 28653 tablet_replica.cc:330] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.898562 28653 raft_consensus.cc:2229] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.898547 28655 raft_consensus.cc:348] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.898607 28655 raft_consensus.cc:374] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.898626 28655 raft_consensus.cc:729] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.898643 28653 pending_rounds.cc:62] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:15.898659 29181 mvcc.cc:205] Tried to move back new op lower bound from 7023420030464245760 to 7023420029734318080. Current Snapshot: MvccSnapshot[applied={T|T < 7023420030464245760}]
I20240503 02:09:15.898663 28655 consensus_queue.cc:260] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.898710 28706 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.898748 28655 ts_tablet_manager.cc:1398] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.898777 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_8f3497b3f2fd46abb20e2efdefdefb34 [id=7a32c89ef97847cca4b48e7891d95a89]). Expected version 1 got 0
I20240503 02:09:15.898794 28655 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.899070 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet f8419275f80940f4aad5ebc259cb52fc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.899123 28655 tablet_bootstrap.cc:654] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.899135 28654 tablet_replica.cc:330] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.899179 28654 raft_consensus.cc:2229] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.899206 28654 raft_consensus.cc:2258] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.899468 28661 ts_tablet_manager.cc:1888] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.899482 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4dabf28ec9f14b5998910b185852b22c
I20240503 02:09:15.899510 28661 log.cc:1196] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/42dc7b64b7d4484296de6b8fc78e2705
I20240503 02:09:15.899574 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet c993bef72a984db794e4cad2aca7af72 with delete_type TABLET_DATA_DELETED (replaced by tablet 4782d97b11e04c0ba788b9af68ab8802) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.899683 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dabf28ec9f14b5998910b185852b22c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.899773 28661 ts_tablet_manager.cc:1909] T 42dc7b64b7d4484296de6b8fc78e2705 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.899798 28410 raft_consensus.cc:482] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.899801 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dabf28ec9f14b5998910b185852b22c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.899849 28410 raft_consensus.cc:504] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.899940 28410 leader_election.cc:288] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.900425 28710 tablet.cc:1722] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.900477 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eb4d0cbce545c8ad87883b9be0f6cb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.900507 28661 tablet_replica.cc:330] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.900532 27720 raft_consensus.cc:2454] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.900563 28661 raft_consensus.cc:2229] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.900571 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 42dc7b64b7d4484296de6b8fc78e2705 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:15.900590 28661 raft_consensus.cc:2258] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.900647 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eb4d0cbce545c8ad87883b9be0f6cb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.900696 27453 raft_consensus.cc:2454] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.900766 28661 ts_tablet_manager.cc:1875] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.900899 27524 leader_election.cc:302] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.900981 29241 raft_consensus.cc:2781] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.901015 29241 raft_consensus.cc:482] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.901039 29241 raft_consensus.cc:3037] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.901232 28654 ts_tablet_manager.cc:1875] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.901721 29241 raft_consensus.cc:504] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.901820 29241 leader_election.cc:288] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.902240 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dabf28ec9f14b5998910b185852b22c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.902418 29172 tablet.cc:1722] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.902699 28661 ts_tablet_manager.cc:1888] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.902801 28661 log.cc:1196] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f8419275f80940f4aad5ebc259cb52fc
I20240503 02:09:15.903004 28661 ts_tablet_manager.cc:1909] T f8419275f80940f4aad5ebc259cb52fc P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.903206 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 80e4ce547683468a9cb0e1683b248508
I20240503 02:09:15.903273 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 80e4ce547683468a9cb0e1683b248508 deleted and replaced by tablet 23129af1b17c45e093e85c2b0032f864
I20240503 02:09:15.898667 28653 pending_rounds.cc:68] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420029593743360 op_type: NO_OP noop_request { }
I20240503 02:09:15.903316 28653 raft_consensus.cc:2866] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:15.903338 28653 raft_consensus.cc:2258] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.903335 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 78e562fa324448f788adaa4f60da3076 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.903461 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e562fa324448f788adaa4f60da3076. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.903504 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f8419275f80940f4aad5ebc259cb52fc (table default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]) successfully deleted
I20240503 02:09:15.903543 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 78e562fa324448f788adaa4f60da3076 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.903636 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e562fa324448f788adaa4f60da3076. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.904125 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eb4d0cbce545c8ad87883b9be0f6cb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.904178 27720 raft_consensus.cc:3037] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.903352 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 9b66e153765c42128c4f64f0894fbe87 (DEFAULT_TABLE table=default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.904508 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b66e153765c42128c4f64f0894fbe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.904632 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e4ce547683468a9cb0e1683b248508 with delete_type TABLET_DATA_DELETED (replaced by tablet 23129af1b17c45e093e85c2b0032f864) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:15.904644 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62eb4d0cbce545c8ad87883b9be0f6cb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.904692 27453 raft_consensus.cc:3037] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.904737 27720 raft_consensus.cc:2454] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.904968 27523 leader_election.cc:302] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.905030 29241 raft_consensus.cc:2781] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.905062 29241 raft_consensus.cc:686] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.905093 29241 consensus_queue.cc:237] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.905205 28698 tablet.cc:1722] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:15.905236 27453 raft_consensus.cc:2454] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.905449 28653 ts_tablet_manager.cc:1875] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.905514 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e4ce547683468a9cb0e1683b248508 with delete_type TABLET_DATA_DELETED (replaced by tablet 23129af1b17c45e093e85c2b0032f864) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:15.905588 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 9b66e153765c42128c4f64f0894fbe87 (DEFAULT_TABLE table=default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.905675 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b66e153765c42128c4f64f0894fbe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.905941 27200 catalog_manager.cc:5484] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.906569 28653 ts_tablet_manager.cc:1888] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.906608 28653 log.cc:1196] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f8419275f80940f4aad5ebc259cb52fc
I20240503 02:09:15.906775 28653 ts_tablet_manager.cc:1909] T f8419275f80940f4aad5ebc259cb52fc P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.906894 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 (DEFAULT_TABLE table=default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.906970 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4782d97b11e04c0ba788b9af68ab8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.908216 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 (DEFAULT_TABLE table=default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.908318 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4782d97b11e04c0ba788b9af68ab8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.908607 28653 tablet_replica.cc:330] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.908721 28653 raft_consensus.cc:2229] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.908885 28654 ts_tablet_manager.cc:1888] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:15.908926 28654 log.cc:1196] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f8419275f80940f4aad5ebc259cb52fc
I20240503 02:09:15.908936 28653 raft_consensus.cc:2258] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.909060 28654 ts_tablet_manager.cc:1909] T f8419275f80940f4aad5ebc259cb52fc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.909189 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f8419275f80940f4aad5ebc259cb52fc (table default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]) successfully deleted
I20240503 02:09:15.909675 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f8419275f80940f4aad5ebc259cb52fc (table default.Table_bd6940d56b554a6096a02a7ac3fb991f [id=9a80cee12c914209b796665c6999abfb]) successfully deleted
I20240503 02:09:15.909925 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 9b66e153765c42128c4f64f0894fbe87 (DEFAULT_TABLE table=default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.909919 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 23129af1b17c45e093e85c2b0032f864 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.910002 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b66e153765c42128c4f64f0894fbe87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.910002 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23129af1b17c45e093e85c2b0032f864. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.910869 28655 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.910905 28655 ts_tablet_manager.cc:1367] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:15.911027 28655 raft_consensus.cc:348] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.911093 28655 raft_consensus.cc:374] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.911111 28655 raft_consensus.cc:729] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.911142 28655 consensus_queue.cc:260] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.911196 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 78e562fa324448f788adaa4f60da3076 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.911214 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3bc72a7c4997486280a2fdcb3bc07c81 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:15.911227 28655 ts_tablet_manager.cc:1398] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.911280 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e562fa324448f788adaa4f60da3076. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.911298 27200 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050] alter complete (version 1)
I20240503 02:09:15.911283 28655 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.911692 28655 tablet_bootstrap.cc:654] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.912250 28664 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.912287 28664 ts_tablet_manager.cc:1367] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.000s
I20240503 02:09:15.912422 27200 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_676914005e65475eac45ad325ccae621 [id=97264f1a599a4a9680a9f284e2633241] alter complete (version 1)
I20240503 02:09:15.912425 28664 raft_consensus.cc:348] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.912485 28664 raft_consensus.cc:374] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.912513 28664 raft_consensus.cc:729] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.912541 28664 consensus_queue.cc:260] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.912619 28664 ts_tablet_manager.cc:1398] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.912668 28664 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.912880 28664 tablet_bootstrap.cc:654] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.913831 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e4ce547683468a9cb0e1683b248508 with delete_type TABLET_DATA_DELETED (replaced by tablet 23129af1b17c45e093e85c2b0032f864) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:15.913810 27202 catalog_manager.cc:5484] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.913913 28766 raft_consensus.cc:482] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.913964 28766 raft_consensus.cc:504] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.914047 28766 leader_election.cc:288] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.914059 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 23129af1b17c45e093e85c2b0032f864 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.914132 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23129af1b17c45e093e85c2b0032f864. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.914213 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811e1aa52cb2458792359e9baebd5e5c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.914270 28654 tablet_replica.cc:330] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.914332 28654 raft_consensus.cc:2229] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.914332 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811e1aa52cb2458792359e9baebd5e5c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.914384 27720 raft_consensus.cc:2454] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.914271 27582 raft_consensus.cc:2454] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.914551 27386 leader_election.cc:302] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.914594 28654 raft_consensus.cc:2258] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.914737 28766 raft_consensus.cc:2781] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.914772 28766 raft_consensus.cc:482] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.914789 28766 raft_consensus.cc:3037] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.915349 28654 ts_tablet_manager.cc:1875] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.915381 28766 raft_consensus.cc:504] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.915486 28766 leader_election.cc:288] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.915573 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 23129af1b17c45e093e85c2b0032f864 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.915642 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811e1aa52cb2458792359e9baebd5e5c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.915689 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23129af1b17c45e093e85c2b0032f864. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.915693 27720 raft_consensus.cc:3037] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.916222 28664 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.916241 27720 raft_consensus.cc:2454] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.916260 28664 ts_tablet_manager.cc:1367] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.916312 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811e1aa52cb2458792359e9baebd5e5c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.916363 27582 raft_consensus.cc:3037] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.916383 28664 raft_consensus.cc:348] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.916426 28664 raft_consensus.cc:374] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.916442 28664 raft_consensus.cc:729] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.916476 28664 consensus_queue.cc:260] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.916553 28664 ts_tablet_manager.cc:1398] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.916606 28664 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.916911 28664 tablet_bootstrap.cc:654] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.916926 27582 raft_consensus.cc:2454] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.915562 28656 tablet_bootstrap.cc:492] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.917024 28656 ts_tablet_manager.cc:1367] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20240503 02:09:15.917028 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 (DEFAULT_TABLE table=default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:15.917114 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4782d97b11e04c0ba788b9af68ab8802. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:15.917148 28656 raft_consensus.cc:348] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.917200 28656 raft_consensus.cc:374] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.917217 28656 raft_consensus.cc:729] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.917253 28656 consensus_queue.cc:260] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.917269 28654 ts_tablet_manager.cc:1888] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.917308 28654 log.cc:1196] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c993bef72a984db794e4cad2aca7af72
I20240503 02:09:15.917352 28656 ts_tablet_manager.cc:1398] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.917413 28656 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.917424 27388 leader_election.cc:302] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.917538 28654 ts_tablet_manager.cc:1909] T c993bef72a984db794e4cad2aca7af72 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.917675 28766 raft_consensus.cc:2781] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.917722 28766 raft_consensus.cc:686] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.917732 28656 tablet_bootstrap.cc:654] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.917755 28766 consensus_queue.cc:237] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.917863 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:15.918411 28654 tablet_replica.cc:330] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.918473 28654 raft_consensus.cc:2229] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.918502 28654 raft_consensus.cc:2258] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.919183 28653 ts_tablet_manager.cc:1875] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.919618 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7] alter complete (version 1)
I20240503 02:09:15.920552 28653 ts_tablet_manager.cc:1888] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.920578 27202 catalog_manager.cc:5484] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.920641 28653 log.cc:1196] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c993bef72a984db794e4cad2aca7af72
I20240503 02:09:15.920940 28653 ts_tablet_manager.cc:1909] T c993bef72a984db794e4cad2aca7af72 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.921084 28654 ts_tablet_manager.cc:1875] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.921298 28653 tablet_replica.cc:330] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.921362 28653 raft_consensus.cc:2229] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.921413 28653 raft_consensus.cc:2258] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.921438 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:15.921743 28653 ts_tablet_manager.cc:1875] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.922470 28654 ts_tablet_manager.cc:1888] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.922514 28661 tablet_replica.cc:330] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.922526 28654 log.cc:1196] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4dabf28ec9f14b5998910b185852b22c
I20240503 02:09:15.922566 28661 raft_consensus.cc:2229] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.922695 28654 ts_tablet_manager.cc:1909] T 4dabf28ec9f14b5998910b185852b22c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.923085 28654 tablet_replica.cc:330] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:15.923177 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4dabf28ec9f14b5998910b185852b22c (table default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]) successfully deleted
I20240503 02:09:15.923219 28661 raft_consensus.cc:2258] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.923373 28653 ts_tablet_manager.cc:1888] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.923401 28654 raft_consensus.cc:2229] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.923424 28653 log.cc:1196] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4dabf28ec9f14b5998910b185852b22c
I20240503 02:09:15.923444 28654 raft_consensus.cc:2258] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.923627 28654 ts_tablet_manager.cc:1875] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.923710 28661 ts_tablet_manager.cc:1875] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.924702 28654 ts_tablet_manager.cc:1888] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.924759 28654 log.cc:1196] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/80e4ce547683468a9cb0e1683b248508
I20240503 02:09:15.924863 28661 ts_tablet_manager.cc:1888] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:15.924924 28654 ts_tablet_manager.cc:1909] T 80e4ce547683468a9cb0e1683b248508 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:15.925034 28653 ts_tablet_manager.cc:1909] T 4dabf28ec9f14b5998910b185852b22c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.925246 28653 tablet_replica.cc:330] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:15.925289 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:15.925374 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4dabf28ec9f14b5998910b185852b22c (table default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]) successfully deleted
I20240503 02:09:15.925401 28653 raft_consensus.cc:2229] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:15.925458 28653 raft_consensus.cc:2258] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.925463 28656 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.925498 28656 ts_tablet_manager.cc:1367] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:15.925545 28661 log.cc:1196] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c993bef72a984db794e4cad2aca7af72
I20240503 02:09:15.925628 28653 ts_tablet_manager.cc:1875] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.925618 28656 raft_consensus.cc:348] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.925666 28656 raft_consensus.cc:374] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.925690 28656 raft_consensus.cc:729] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.925727 28661 ts_tablet_manager.cc:1909] T c993bef72a984db794e4cad2aca7af72 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.925724 28656 consensus_queue.cc:260] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.925801 28656 ts_tablet_manager.cc:1398] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.925868 28656 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.925976 28661 tablet_replica.cc:330] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.925971 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c993bef72a984db794e4cad2aca7af72 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:15.926064 28661 raft_consensus.cc:2229] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.926106 28661 raft_consensus.cc:2258] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.926223 28656 tablet_bootstrap.cc:654] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.926295 28661 ts_tablet_manager.cc:1875] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.926714 28656 tablet_bootstrap.cc:492] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.926764 28656 ts_tablet_manager.cc:1367] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.926891 28656 raft_consensus.cc:348] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.926955 28656 raft_consensus.cc:374] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.926980 28656 raft_consensus.cc:729] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.927013 28656 consensus_queue.cc:260] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.927119 28656 ts_tablet_manager.cc:1398] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.927222 28656 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.927265 28653 ts_tablet_manager.cc:1888] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.927353 28653 log.cc:1196] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/80e4ce547683468a9cb0e1683b248508
I20240503 02:09:15.927568 28656 tablet_bootstrap.cc:654] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.927757 28653 ts_tablet_manager.cc:1909] T 80e4ce547683468a9cb0e1683b248508 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:15.927958 28661 ts_tablet_manager.cc:1888] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:15.928146 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:15.928177 28661 log.cc:1196] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4dabf28ec9f14b5998910b185852b22c
I20240503 02:09:15.928282 28656 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.928326 28656 ts_tablet_manager.cc:1367] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:15.928371 28661 ts_tablet_manager.cc:1909] T 4dabf28ec9f14b5998910b185852b22c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.928557 28656 raft_consensus.cc:348] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.928625 28656 raft_consensus.cc:374] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.928651 28656 raft_consensus.cc:729] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.928689 28656 consensus_queue.cc:260] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.928769 28656 ts_tablet_manager.cc:1398] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.928838 28656 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.928975 28661 tablet_replica.cc:330] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:15.928966 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4dabf28ec9f14b5998910b185852b22c (table default.Table_5ee8aa23d34f421f8f05b0982609667a [id=6b6677b40536460d8883646ae6f6177b]) successfully deleted
I20240503 02:09:15.929080 28661 raft_consensus.cc:2229] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:15.929181 28656 tablet_bootstrap.cc:654] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.929152 28661 raft_consensus.cc:2258] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:15.929700 28656 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.929704 28661 ts_tablet_manager.cc:1875] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:15.929747 28656 ts_tablet_manager.cc:1367] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.929859 28656 raft_consensus.cc:348] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.929917 28656 raft_consensus.cc:374] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.929940 28656 raft_consensus.cc:729] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.929976 28656 consensus_queue.cc:260] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.930078 28656 ts_tablet_manager.cc:1398] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.930161 28656 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.930476 28656 tablet_bootstrap.cc:654] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.930894 28656 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.930941 28656 ts_tablet_manager.cc:1367] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:15.931056 28656 raft_consensus.cc:348] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.931126 28656 raft_consensus.cc:374] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.931151 28656 raft_consensus.cc:729] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.931190 28656 consensus_queue.cc:260] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.931267 28656 ts_tablet_manager.cc:1398] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.931344 28656 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.931665 28656 tablet_bootstrap.cc:654] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.931964 28661 ts_tablet_manager.cc:1888] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:15.932015 28661 log.cc:1196] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/80e4ce547683468a9cb0e1683b248508
I20240503 02:09:15.932117 28664 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.932175 28664 ts_tablet_manager.cc:1367] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.000s
I20240503 02:09:15.932303 28664 raft_consensus.cc:348] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.932374 28664 raft_consensus.cc:374] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.932391 28664 raft_consensus.cc:729] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.932307 28661 ts_tablet_manager.cc:1909] T 80e4ce547683468a9cb0e1683b248508 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:15.932428 28664 consensus_queue.cc:260] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.932514 28664 ts_tablet_manager.cc:1398] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.932585 28664 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.932837 28655 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.932875 28655 ts_tablet_manager.cc:1367] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.000s
I20240503 02:09:15.932895 28664 tablet_bootstrap.cc:654] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.933013 28655 raft_consensus.cc:348] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.933074 28655 raft_consensus.cc:374] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.933095 28655 raft_consensus.cc:729] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.933130 28655 consensus_queue.cc:260] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.933212 28655 ts_tablet_manager.cc:1398] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.933281 28655 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.933423 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 80e4ce547683468a9cb0e1683b248508 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:15.933617 28655 tablet_bootstrap.cc:654] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.934532 28655 tablet_bootstrap.cc:492] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.934573 28655 ts_tablet_manager.cc:1367] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.934692 28655 raft_consensus.cc:348] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.934743 28655 raft_consensus.cc:374] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.934759 28655 raft_consensus.cc:729] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.934792 28655 consensus_queue.cc:260] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.934866 28655 ts_tablet_manager.cc:1398] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.934926 28655 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.935062 28656 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.935103 28656 ts_tablet_manager.cc:1367] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:15.935209 28656 raft_consensus.cc:348] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.935245 28655 tablet_bootstrap.cc:654] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.935263 28656 raft_consensus.cc:374] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.935281 28656 raft_consensus.cc:729] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.935317 28656 consensus_queue.cc:260] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.935390 28656 ts_tablet_manager.cc:1398] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.935458 28656 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.935811 28656 tablet_bootstrap.cc:654] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.936247 28655 tablet_bootstrap.cc:492] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.936290 28655 ts_tablet_manager.cc:1367] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.936427 28655 raft_consensus.cc:348] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.936475 28655 raft_consensus.cc:374] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.936492 28655 raft_consensus.cc:729] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.936522 28655 consensus_queue.cc:260] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.936599 28655 ts_tablet_manager.cc:1398] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.936642 28655 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.936949 28655 tablet_bootstrap.cc:654] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.937772 28664 tablet_bootstrap.cc:492] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.937813 28664 ts_tablet_manager.cc:1367] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.937940 28655 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.937928 28664 raft_consensus.cc:348] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.937978 28664 raft_consensus.cc:374] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.937976 28655 ts_tablet_manager.cc:1367] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.937996 28664 raft_consensus.cc:729] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.938028 28664 consensus_queue.cc:260] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.938099 28664 ts_tablet_manager.cc:1398] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.938088 28655 raft_consensus.cc:348] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.938139 28655 raft_consensus.cc:374] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.938155 28655 raft_consensus.cc:729] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.938158 28664 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.938189 28655 consensus_queue.cc:260] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.938261 28655 ts_tablet_manager.cc:1398] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.938306 28655 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.938455 28664 tablet_bootstrap.cc:654] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.938627 28655 tablet_bootstrap.cc:654] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.939543 28656 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.939585 28656 ts_tablet_manager.cc:1367] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.939704 28656 raft_consensus.cc:348] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.939754 28656 raft_consensus.cc:374] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.939770 28656 raft_consensus.cc:729] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.939800 28656 consensus_queue.cc:260] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.939882 28656 ts_tablet_manager.cc:1398] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.939949 28656 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.940233 28655 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.940241 28656 tablet_bootstrap.cc:654] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.940274 28655 ts_tablet_manager.cc:1367] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.940394 28655 raft_consensus.cc:348] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.940449 28655 raft_consensus.cc:374] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.940467 28655 raft_consensus.cc:729] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.940500 28655 consensus_queue.cc:260] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.940585 28655 ts_tablet_manager.cc:1398] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.940629 28655 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.941035 28655 tablet_bootstrap.cc:654] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.941942 28655 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.941987 28655 ts_tablet_manager.cc:1367] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.942101 28655 raft_consensus.cc:348] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.942142 28655 raft_consensus.cc:374] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.942152 28655 raft_consensus.cc:729] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.942183 28655 consensus_queue.cc:260] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.942251 28655 ts_tablet_manager.cc:1398] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.942297 28655 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.942636 28655 tablet_bootstrap.cc:654] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.943192 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
W20240503 02:09:15.943266 27202 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4782d97b11e04c0ba788b9af68ab8802 consensus state has no leader
I20240503 02:09:15.943478 28664 tablet_bootstrap.cc:492] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.943519 28664 ts_tablet_manager.cc:1367] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.943599 28664 raft_consensus.cc:348] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.943647 28664 raft_consensus.cc:374] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.943665 28664 raft_consensus.cc:729] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.943701 28664 consensus_queue.cc:260] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.943778 28664 ts_tablet_manager.cc:1398] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.943842 28664 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.943869 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.944052 28664 tablet_bootstrap.cc:654] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.944379 28655 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.944418 28655 ts_tablet_manager.cc:1367] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.944545 28655 raft_consensus.cc:348] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.944586 28655 raft_consensus.cc:374] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.944599 28655 raft_consensus.cc:729] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.944638 28655 consensus_queue.cc:260] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.944710 28655 ts_tablet_manager.cc:1398] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.944751 28655 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.945026 28655 tablet_bootstrap.cc:654] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.945443 28656 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.945478 28656 ts_tablet_manager.cc:1367] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:15.945585 28656 raft_consensus.cc:348] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.945626 28656 raft_consensus.cc:374] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.945641 28656 raft_consensus.cc:729] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.945685 28656 consensus_queue.cc:260] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.945760 28656 ts_tablet_manager.cc:1398] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.945813 28656 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.946116 28656 tablet_bootstrap.cc:654] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.946720 28656 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.946763 28656 ts_tablet_manager.cc:1367] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.946870 28656 raft_consensus.cc:348] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.946941 28656 raft_consensus.cc:374] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.946960 28656 raft_consensus.cc:729] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.946992 28656 consensus_queue.cc:260] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.947077 28656 ts_tablet_manager.cc:1398] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.947160 28656 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:15.947150 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.947248 28655 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.947465 28656 tablet_bootstrap.cc:654] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.947794 28655 ts_tablet_manager.cc:1367] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:15.947927 28655 raft_consensus.cc:348] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.947984 28655 raft_consensus.cc:374] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.948002 28655 raft_consensus.cc:729] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.948052 28655 consensus_queue.cc:260] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.948400 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.948441 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
W20240503 02:09:15.948493 27202 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 9b66e153765c42128c4f64f0894fbe87 consensus state has no leader
I20240503 02:09:15.948705 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:15.948817 28656 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:15.948856 28656 ts_tablet_manager.cc:1367] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.948995 28656 raft_consensus.cc:348] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.949047 28656 raft_consensus.cc:374] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.949065 28656 raft_consensus.cc:729] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.949117 28656 consensus_queue.cc:260] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.949229 28664 tablet_bootstrap.cc:492] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.949265 28664 ts_tablet_manager.cc:1367] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:15.949352 28656 ts_tablet_manager.cc:1398] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.949401 28664 raft_consensus.cc:348] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.949451 28664 raft_consensus.cc:374] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.949468 28664 raft_consensus.cc:729] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.949494 28664 consensus_queue.cc:260] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.949527 28655 ts_tablet_manager.cc:1398] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:15.949584 28664 ts_tablet_manager.cc:1398] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.949645 28664 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.949961 28664 tablet_bootstrap.cc:654] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.950363 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
W20240503 02:09:15.950446 27200 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 23129af1b17c45e093e85c2b0032f864 consensus state has no leader
I20240503 02:09:15.950894 28774 raft_consensus.cc:482] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.950945 28774 raft_consensus.cc:504] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.951057 28774 leader_election.cc:288] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.951200 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c4d03198554e4aa7a11125f851debc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.951252 27582 raft_consensus.cc:2454] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.951359 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:15.951370 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c4d03198554e4aa7a11125f851debc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.951440 27453 raft_consensus.cc:2454] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.951579 27659 leader_election.cc:302] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.951709 28774 raft_consensus.cc:2781] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.951736 28774 raft_consensus.cc:482] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.951752 28774 raft_consensus.cc:3037] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.951849 28655 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:15.951886 28664 tablet_bootstrap.cc:492] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.951916 28664 ts_tablet_manager.cc:1367] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.951982 28664 raft_consensus.cc:348] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.952020 28664 raft_consensus.cc:374] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.952035 28664 raft_consensus.cc:729] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.952068 28664 consensus_queue.cc:260] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.952150 28664 ts_tablet_manager.cc:1398] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.952307 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:15.952487 28655 tablet_bootstrap.cc:654] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.953428 28655 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:15.953476 28655 ts_tablet_manager.cc:1367] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:15.953615 28655 raft_consensus.cc:348] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.953670 28655 raft_consensus.cc:374] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.953687 28655 raft_consensus.cc:729] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.953732 28655 consensus_queue.cc:260] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.953817 28655 ts_tablet_manager.cc:1398] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.952224 28664 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.954330 28664 tablet_bootstrap.cc:654] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.952334 28774 raft_consensus.cc:504] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.954599 28774 leader_election.cc:288] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.954869 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c4d03198554e4aa7a11125f851debc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.954874 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34c4d03198554e4aa7a11125f851debc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.954936 27582 raft_consensus.cc:3037] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.954936 27453 raft_consensus.cc:3037] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.955093 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:15.955559 27582 raft_consensus.cc:2454] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.955569 27453 raft_consensus.cc:2454] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.955639 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:15.955718 27656 leader_election.cc:302] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.955794 28774 raft_consensus.cc:2781] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.955842 28774 raft_consensus.cc:686] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:15.955878 28774 consensus_queue.cc:237] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.956200 28664 tablet_bootstrap.cc:492] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.956243 28664 ts_tablet_manager.cc:1367] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:15.956321 28664 raft_consensus.cc:348] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.956368 28664 raft_consensus.cc:374] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.956384 28664 raft_consensus.cc:729] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.956419 28664 consensus_queue.cc:260] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.956492 28664 ts_tablet_manager.cc:1398] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.956542 28664 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.956740 28664 tablet_bootstrap.cc:654] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.957053 27202 catalog_manager.cc:5484] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.957213 28664 tablet_bootstrap.cc:492] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.957269 28664 ts_tablet_manager.cc:1367] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.957425 28664 raft_consensus.cc:348] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.957486 28664 raft_consensus.cc:374] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.957504 28664 raft_consensus.cc:729] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.957549 28664 consensus_queue.cc:260] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.957635 28664 ts_tablet_manager.cc:1398] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.957682 28664 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:15.958019 28664 tablet_bootstrap.cc:654] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:15.958853 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:15.958899 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:15.958979 28664 tablet_bootstrap.cc:492] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:15.959080 28664 ts_tablet_manager.cc:1367] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:15.959214 28664 raft_consensus.cc:348] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.959256 28664 raft_consensus.cc:374] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:15.959268 28664 raft_consensus.cc:729] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:15.959303 28664 consensus_queue.cc:260] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.959383 28664 ts_tablet_manager.cc:1398] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:15.959872 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.967056 28685 raft_consensus.cc:482] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.967105 28685 raft_consensus.cc:504] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.967211 28685 leader_election.cc:288] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.967348 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d4a1909677544549029137b79501235" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.967348 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d4a1909677544549029137b79501235" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.967404 27582 raft_consensus.cc:2454] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.967406 27720 raft_consensus.cc:2454] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:15.967545 27386 leader_election.cc:302] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.967639 28685 raft_consensus.cc:2781] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.967677 28685 raft_consensus.cc:482] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.967696 28685 raft_consensus.cc:3037] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.968154 28685 raft_consensus.cc:504] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.968258 28685 leader_election.cc:288] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.968384 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d4a1909677544549029137b79501235" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.968420 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d4a1909677544549029137b79501235" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.968447 27582 raft_consensus.cc:3037] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.968468 27720 raft_consensus.cc:3037] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.968894 27582 raft_consensus.cc:2454] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.968910 27720 raft_consensus.cc:2454] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:15.969048 27386 leader_election.cc:302] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.969134 28685 raft_consensus.cc:2781] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.969179 28685 raft_consensus.cc:686] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:15.969213 28685 consensus_queue.cc:237] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.969913 27202 catalog_manager.cc:5484] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.970016 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 0d4a1909677544549029137b79501235 (table default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81]). Expected version 1 got 0
I20240503 02:09:15.975685 28410 consensus_queue.cc:1035] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.976661 28410 raft_consensus.cc:482] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.976708 28410 raft_consensus.cc:504] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.976730 28507 consensus_queue.cc:1035] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:15.976794 27202 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:15.976799 28410 leader_election.cc:288] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.977301 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4782d97b11e04c0ba788b9af68ab8802" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.977404 27719 raft_consensus.cc:2454] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.977535 27523 leader_election.cc:302] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.977600 29241 raft_consensus.cc:2781] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.977630 29241 raft_consensus.cc:482] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.977646 29241 raft_consensus.cc:3037] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.978277 29241 raft_consensus.cc:504] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.978394 29241 leader_election.cc:288] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.978814 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4782d97b11e04c0ba788b9af68ab8802" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.978826 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4782d97b11e04c0ba788b9af68ab8802" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.978881 27720 raft_consensus.cc:3037] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.978883 27450 raft_consensus.cc:2454] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.979458 27720 raft_consensus.cc:2454] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.979669 27523 leader_election.cc:302] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.979663 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4782d97b11e04c0ba788b9af68ab8802" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.979720 27450 raft_consensus.cc:3037] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.979743 29241 raft_consensus.cc:2781] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.979782 29241 raft_consensus.cc:686] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.979820 29241 consensus_queue.cc:237] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:15.980314 27450 raft_consensus.cc:2454] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.980463 27202 catalog_manager.cc:5484] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:15.980566 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:15.981500 29240 raft_consensus.cc:482] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.981559 29240 raft_consensus.cc:504] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.981688 29240 leader_election.cc:288] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.981770 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e562fa324448f788adaa4f60da3076" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.981834 27450 raft_consensus.cc:2454] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.982009 27524 leader_election.cc:302] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.982079 29240 raft_consensus.cc:2781] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.982117 29240 raft_consensus.cc:482] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.982133 29240 raft_consensus.cc:3037] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.982611 29240 raft_consensus.cc:504] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.982718 29240 leader_election.cc:288] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.982834 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e562fa324448f788adaa4f60da3076" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.982851 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e562fa324448f788adaa4f60da3076" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.982901 27450 raft_consensus.cc:3037] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.982892 27720 raft_consensus.cc:2454] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.982897 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78e562fa324448f788adaa4f60da3076" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.982967 27719 raft_consensus.cc:3037] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.983378 27450 raft_consensus.cc:2454] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.983433 27719 raft_consensus.cc:2454] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.983531 27524 leader_election.cc:302] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.983613 29240 raft_consensus.cc:2781] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.983657 29240 raft_consensus.cc:686] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.983721 29240 consensus_queue.cc:237] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.984445 27202 catalog_manager.cc:5484] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.985438 28740 consensus_queue.cc:1035] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.986994 28693 consensus_queue.cc:1035] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.990404 28766 consensus_queue.cc:1035] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.991629 28409 consensus_queue.cc:1035] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.993978 29241 raft_consensus.cc:482] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.994055 29241 raft_consensus.cc:504] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.994189 29241 leader_election.cc:288] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.994330 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c35a98c21b848e79a38f1b9cf5919a6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.994401 27450 raft_consensus.cc:2454] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.994544 27524 leader_election.cc:302] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.994661 29241 raft_consensus.cc:2781] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.994707 29241 raft_consensus.cc:482] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.994730 29241 raft_consensus.cc:3037] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.994810 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c35a98c21b848e79a38f1b9cf5919a6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:15.994871 27719 raft_consensus.cc:2454] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:15.995195 29241 raft_consensus.cc:504] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.995296 29241 leader_election.cc:288] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:15.995405 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c35a98c21b848e79a38f1b9cf5919a6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:15.995407 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c35a98c21b848e79a38f1b9cf5919a6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.995451 27450 raft_consensus.cc:3037] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.995453 27719 raft_consensus.cc:3037] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.995860 27719 raft_consensus.cc:2454] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.995882 27450 raft_consensus.cc:2454] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:15.996004 27524 leader_election.cc:302] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:15.996093 29241 raft_consensus.cc:2781] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:15.996132 29241 raft_consensus.cc:686] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:15.996165 29241 consensus_queue.cc:237] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.996793 27202 catalog_manager.cc:5484] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:15.997684 28693 raft_consensus.cc:482] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:15.997735 28693 raft_consensus.cc:504] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.997838 28693 leader_election.cc:288] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.997969 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a4f79dd7ac4d929abda2d67aad87d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:15.998034 27450 raft_consensus.cc:2454] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.998054 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a4f79dd7ac4d929abda2d67aad87d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:15.998102 27582 raft_consensus.cc:2454] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:15.998225 27659 leader_election.cc:302] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:15.998315 28693 raft_consensus.cc:2781] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:15.998350 28693 raft_consensus.cc:482] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:15.998368 28693 raft_consensus.cc:3037] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.998432 27719 raft_consensus.cc:1261] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.998572 29241 consensus_queue.cc:1035] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.998916 28693 raft_consensus.cc:504] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:15.999022 28693 leader_election.cc:288] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:15.999177 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a4f79dd7ac4d929abda2d67aad87d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:15.999235 27582 raft_consensus.cc:3037] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.999413 27450 raft_consensus.cc:1261] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:15.999491 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a4f79dd7ac4d929abda2d67aad87d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:15.999524 27453 raft_consensus.cc:3037] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:15.999799 27582 raft_consensus.cc:2454] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:15.999866 29240 consensus_queue.cc:1035] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:15.999905 27656 leader_election.cc:302] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.000089 28693 raft_consensus.cc:2781] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.000136 28693 raft_consensus.cc:686] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.000115 27453 raft_consensus.cc:2454] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.000171 28693 consensus_queue.cc:237] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.000219 28698 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.000391 28710 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.000809 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:16.001008 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 1 got 0
I20240503 02:09:16.001467 28706 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.002707 27200 catalog_manager.cc:5484] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.004154 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1193de5d35994a1cbc3bd480ac42bd2f" } modify_external_catalogs: true
I20240503 02:09:16.004222 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1193de5d35994a1cbc3bd480ac42bd2f" } modify_external_catalogs: true
I20240503 02:09:16.005287 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9fe7dc7197554c6ab1604d275eb67e8b
I20240503 02:09:16.005534 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.005533 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.005617 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe7dc7197554c6ab1604d275eb67e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:15 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.005620 28653 tablet_replica.cc:330] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.005684 28653 raft_consensus.cc:2229] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.005689 28654 tablet_replica.cc:330] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.005689 28661 tablet_replica.cc:330] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.005720 28653 raft_consensus.cc:2258] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.005743 28654 raft_consensus.cc:2229] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.005743 28661 raft_consensus.cc:2229] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.005774 28654 raft_consensus.cc:2258] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.005797 28661 raft_consensus.cc:2258] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.005923 28653 ts_tablet_manager.cc:1875] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.006038 28661 ts_tablet_manager.cc:1875] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.006127 28654 ts_tablet_manager.cc:1875] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.007143 28654 ts_tablet_manager.cc:1888] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.007186 28654 log.cc:1196] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9fe7dc7197554c6ab1604d275eb67e8b
I20240503 02:09:16.007239 28653 ts_tablet_manager.cc:1888] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.007239 28661 ts_tablet_manager.cc:1888] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.007284 28661 log.cc:1196] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9fe7dc7197554c6ab1604d275eb67e8b
I20240503 02:09:16.007284 28653 log.cc:1196] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9fe7dc7197554c6ab1604d275eb67e8b
I20240503 02:09:16.007325 28654 ts_tablet_manager.cc:1909] T 9fe7dc7197554c6ab1604d275eb67e8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.007469 28661 ts_tablet_manager.cc:1909] T 9fe7dc7197554c6ab1604d275eb67e8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.007469 28653 ts_tablet_manager.cc:1909] T 9fe7dc7197554c6ab1604d275eb67e8b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.007751 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]) successfully deleted
I20240503 02:09:16.008096 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]) successfully deleted
I20240503 02:09:16.008209 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9fe7dc7197554c6ab1604d275eb67e8b (table default.Table_1193de5d35994a1cbc3bd480ac42bd2f [id=2921ee2db01a428eb4718fb9c08038bf]) successfully deleted
I20240503 02:09:16.021025 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_03f97ede69bd47fdb581e34a506b8dea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.021136 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03f97ede69bd47fdb581e34a506b8dea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.022251 27853 consensus_queue.cc:1035] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.024399 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a682a7477de54349b2021f1d660a7d1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.024495 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a682a7477de54349b2021f1d660a7d1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.024777 28740 consensus_queue.cc:1035] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.029951 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9eebf349071044e7a8b8207e8e7dad3e" } modify_external_catalogs: true
I20240503 02:09:16.030019 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9eebf349071044e7a8b8207e8e7dad3e" } modify_external_catalogs: true
I20240503 02:09:16.031167 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 503bd37c6def464496f6dd4489fca009
I20240503 02:09:16.031421 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 503bd37c6def464496f6dd4489fca009 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.031526 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet 503bd37c6def464496f6dd4489fca009 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.031621 28654 tablet_replica.cc:330] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.031678 28654 raft_consensus.cc:2229] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.031713 28654 raft_consensus.cc:2258] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.031824 28661 tablet_replica.cc:330] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.031888 28661 raft_consensus.cc:2229] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.031945 28661 raft_consensus.cc:2258] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.031977 28654 ts_tablet_manager.cc:1875] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.032194 28661 ts_tablet_manager.cc:1875] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.033138 28654 ts_tablet_manager.cc:1888] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.033185 28654 log.cc:1196] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/503bd37c6def464496f6dd4489fca009
I20240503 02:09:16.033238 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 503bd37c6def464496f6dd4489fca009 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.033296 28661 ts_tablet_manager.cc:1888] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.033375 28653 tablet_replica.cc:330] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.033390 28661 log.cc:1196] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/503bd37c6def464496f6dd4489fca009
I20240503 02:09:16.033401 28654 ts_tablet_manager.cc:1909] T 503bd37c6def464496f6dd4489fca009 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.033429 28653 raft_consensus.cc:2229] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.033463 28653 raft_consensus.cc:2258] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.033589 28661 ts_tablet_manager.cc:1909] T 503bd37c6def464496f6dd4489fca009 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.033658 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 503bd37c6def464496f6dd4489fca009 (table default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]) successfully deleted
I20240503 02:09:16.033684 28653 ts_tablet_manager.cc:1875] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.033910 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 503bd37c6def464496f6dd4489fca009 (table default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]) successfully deleted
I20240503 02:09:16.034089 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1db3284882814b5a817c185b2ba87962"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.034178 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1db3284882814b5a817c185b2ba87962 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.034809 28653 ts_tablet_manager.cc:1888] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.034863 28653 log.cc:1196] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/503bd37c6def464496f6dd4489fca009
I20240503 02:09:16.035051 28653 ts_tablet_manager.cc:1909] T 503bd37c6def464496f6dd4489fca009 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.035017 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9ae759f2c210464499b9b30fa848dbcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.035107 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ae759f2c210464499b9b30fa848dbcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.035562 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 503bd37c6def464496f6dd4489fca009 (table default.Table_9eebf349071044e7a8b8207e8e7dad3e [id=cf3637b2a563413cb2d2c2d282386961]) successfully deleted
I20240503 02:09:16.036476 28409 raft_consensus.cc:482] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.036528 28409 raft_consensus.cc:504] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.036628 28409 leader_election.cc:288] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.036754 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ce961ecae44ca18642f64cf3f02977" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.036821 27582 raft_consensus.cc:2454] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.036890 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ce961ecae44ca18642f64cf3f02977" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.036949 27719 raft_consensus.cc:2454] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.036948 27386 leader_election.cc:302] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.037019 28409 raft_consensus.cc:2781] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.037066 28409 raft_consensus.cc:482] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.037086 28409 raft_consensus.cc:3037] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.037719 28409 raft_consensus.cc:504] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.037828 28409 leader_election.cc:288] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.037992 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ce961ecae44ca18642f64cf3f02977" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.038059 27582 raft_consensus.cc:3037] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.038611 27582 raft_consensus.cc:2454] T a8ce961ecae44ca18642f64cf3f02977 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.038695 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ce961ecae44ca18642f64cf3f02977" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.038748 27719 raft_consensus.cc:3037] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.038750 27386 leader_election.cc:302] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.038831 28409 raft_consensus.cc:2781] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.038866 28409 raft_consensus.cc:686] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.038900 28409 consensus_queue.cc:237] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.039274 27719 raft_consensus.cc:2454] T a8ce961ecae44ca18642f64cf3f02977 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.039763 27195 catalog_manager.cc:5484] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.043471 27848 consensus_queue.cc:1035] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.044209 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 (DEFAULT_TABLE table=default.Table_1db3284882814b5a817c185b2ba87962 [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.044279 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.044338 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f7c3e3e6e643948b8668c675a20b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.044900 28507 consensus_queue.cc:1035] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.045006 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb685fcf4a045f99b2cdebe5bf164d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.045449 28664 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.044210 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 78330603666e4171845cf14dade17795 (DEFAULT_TABLE table=default.Table_a682a7477de54349b2021f1d660a7d1f [id=b743de8e555f4387b6ec8bd798231c72]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.045714 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78330603666e4171845cf14dade17795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.045701 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.045791 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb685fcf4a045f99b2cdebe5bf164d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.045867 28664 tablet_bootstrap.cc:654] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.046092 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 78330603666e4171845cf14dade17795 (DEFAULT_TABLE table=default.Table_a682a7477de54349b2021f1d660a7d1f [id=b743de8e555f4387b6ec8bd798231c72]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.046175 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78330603666e4171845cf14dade17795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.046209 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 (DEFAULT_TABLE table=default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.046294 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e86a1d93fb64511aa9def6f04e05b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.046721 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 (DEFAULT_TABLE table=default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.046800 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e86a1d93fb64511aa9def6f04e05b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.046896 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 (DEFAULT_TABLE table=default.Table_1db3284882814b5a817c185b2ba87962 [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.046965 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f7c3e3e6e643948b8668c675a20b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.047188 28656 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.047535 28656 tablet_bootstrap.cc:654] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.048408 28656 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.048450 28656 ts_tablet_manager.cc:1367] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.048570 28656 raft_consensus.cc:348] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.048632 28656 raft_consensus.cc:374] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.048655 28656 raft_consensus.cc:729] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.048689 28656 consensus_queue.cc:260] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.048816 28656 ts_tablet_manager.cc:1398] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:16.048874 28656 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.049147 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 78330603666e4171845cf14dade17795 (DEFAULT_TABLE table=default.Table_a682a7477de54349b2021f1d660a7d1f [id=b743de8e555f4387b6ec8bd798231c72]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.049147 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0ed3bba2c51d455792d4ae75226349a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.049232 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ed3bba2c51d455792d4ae75226349a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.049242 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78330603666e4171845cf14dade17795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.049232 28656 tablet_bootstrap.cc:654] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.049778 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 (DEFAULT_TABLE table=default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.049866 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e86a1d93fb64511aa9def6f04e05b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.050172 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_00d4eb75b1d141c59384c55a3438531e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.050223 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6dea9657852d452dbf25d95e393bbd74" } new_table_name: "default.Table_63bcae89d666400790b5517d39e533ca" modify_external_catalogs: true
W20240503 02:09:16.050262 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00d4eb75b1d141c59384c55a3438531e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.050395 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a332ac50155641dc87ea83b511517566"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.050417 28655 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:16.050460 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a332ac50155641dc87ea83b511517566 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.050767 28655 tablet_bootstrap.cc:654] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.050880 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_11b878bc4a1740f2ab615579a62a5cfd" } modify_external_catalogs: true
I20240503 02:09:16.050944 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_11b878bc4a1740f2ab615579a62a5cfd" } modify_external_catalogs: true
I20240503 02:09:16.051105 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.051195 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fb685fcf4a045f99b2cdebe5bf164d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.051565 28655 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.051604 28655 ts_tablet_manager.cc:1367] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.051718 28655 raft_consensus.cc:348] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.051766 28655 raft_consensus.cc:374] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.051784 28655 raft_consensus.cc:729] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.051818 28655 consensus_queue.cc:260] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.051913 28655 ts_tablet_manager.cc:1398] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.052219 28664 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.052256 28664 ts_tablet_manager.cc:1367] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:16.052263 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 (DEFAULT_TABLE table=default.Table_1db3284882814b5a817c185b2ba87962 [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.052336 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36f7c3e3e6e643948b8668c675a20b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.052356 28655 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.052381 28664 raft_consensus.cc:348] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.052428 28664 raft_consensus.cc:374] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.052448 28664 raft_consensus.cc:729] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.052485 28664 consensus_queue.cc:260] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.052578 28664 ts_tablet_manager.cc:1398] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.052625 28664 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.052723 28655 tablet_bootstrap.cc:654] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.052964 28664 tablet_bootstrap.cc:654] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.053464 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f496c1bdb19a424a9ef4d1f772825869"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.053551 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f496c1bdb19a424a9ef4d1f772825869 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.054314 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d4ff0e8769f94e43a73d0dde16f9e531"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.054399 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4ff0e8769f94e43a73d0dde16f9e531 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.054822 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet bdc8839c6cb14c648d15ef51ccb722ea
I20240503 02:09:16.055418 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.055485 28654 tablet_replica.cc:330] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.055536 28654 raft_consensus.cc:2229] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.055562 28654 raft_consensus.cc:2258] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.055608 27450 raft_consensus.cc:1261] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.055619 27691 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.055666 28654 ts_tablet_manager.cc:1875] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.055716 27582 raft_consensus.cc:1261] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.055763 28774 consensus_queue.cc:1035] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.055804 28661 tablet_replica.cc:330] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.055855 28661 raft_consensus.cc:2229] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.055873 28774 consensus_queue.cc:1035] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.056697 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc8839c6cb14c648d15ef51ccb722ea with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.056789 28653 tablet_replica.cc:330] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.056803 28654 ts_tablet_manager.cc:1888] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.056838 28653 raft_consensus.cc:2229] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.056846 28654 log.cc:1196] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/bdc8839c6cb14c648d15ef51ccb722ea
I20240503 02:09:16.056871 27587 raft_consensus.cc:1261] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.056901 28653 raft_consensus.cc:2258] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.057006 28653 ts_tablet_manager.cc:1875] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.057021 28654 ts_tablet_manager.cc:1909] T bdc8839c6cb14c648d15ef51ccb722ea P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.057021 27719 raft_consensus.cc:1261] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.057160 28409 consensus_queue.cc:1035] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.057364 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet fd4021dcc28f40fdb505f2491e034a40 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.057456 28409 consensus_queue.cc:1035] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.057523 28685 consensus_queue.cc:1035] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.058233 28653 ts_tablet_manager.cc:1888] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.058282 28653 log.cc:1196] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/bdc8839c6cb14c648d15ef51ccb722ea
I20240503 02:09:16.058458 28653 ts_tablet_manager.cc:1909] T bdc8839c6cb14c648d15ef51ccb722ea P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.058701 27565 tablet_service.cc:1467] Processing CreateTablet for tablet ef574442d0ca4988b77635dd7e8ce5db (DEFAULT_TABLE table=default.Table_0ed3bba2c51d455792d4ae75226349a1 [id=db1c5cc1db874e75a028507343d103a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.058770 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef574442d0ca4988b77635dd7e8ce5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.059154 27691 tablet_service.cc:1467] Processing CreateTablet for tablet ef574442d0ca4988b77635dd7e8ce5db (DEFAULT_TABLE table=default.Table_0ed3bba2c51d455792d4ae75226349a1 [id=db1c5cc1db874e75a028507343d103a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.059242 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef574442d0ca4988b77635dd7e8ce5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.055884 28661 raft_consensus.cc:2258] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.059896 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet bdc8839c6cb14c648d15ef51ccb722ea (table default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]) successfully deleted
I20240503 02:09:16.059937 28661 ts_tablet_manager.cc:1875] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.060640 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet bdc8839c6cb14c648d15ef51ccb722ea (table default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]) successfully deleted
I20240503 02:09:16.061039 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 138ffe8057904a2485289b61064bf382 (table default.Table_63bcae89d666400790b5517d39e533ca [id=cd9e7541be36427ea8e01bbba147d869]). Expected version 1 got 0
I20240503 02:09:16.061092 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 0d4a1909677544549029137b79501235 (table default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81]). Expected version 1 got 0
I20240503 02:09:16.061295 27433 tablet_service.cc:1467] Processing CreateTablet for tablet ef574442d0ca4988b77635dd7e8ce5db (DEFAULT_TABLE table=default.Table_0ed3bba2c51d455792d4ae75226349a1 [id=db1c5cc1db874e75a028507343d103a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.061422 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef574442d0ca4988b77635dd7e8ce5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.061878 28664 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.061918 28664 ts_tablet_manager.cc:1367] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:16.062036 28664 raft_consensus.cc:348] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.062078 28664 raft_consensus.cc:374] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.062094 28664 raft_consensus.cc:729] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.062129 28664 consensus_queue.cc:260] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.062203 28664 ts_tablet_manager.cc:1398] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.062252 28664 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.062667 28664 tablet_bootstrap.cc:654] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.062928 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet fd4021dcc28f40fdb505f2491e034a40
I20240503 02:09:16.062947 27426 tablet_service.cc:1467] Processing CreateTablet for tablet c9b402f1be4d4fab80a0cef3392725bf (DEFAULT_TABLE table=default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.062994 27195 catalog_manager.cc:6554] ReplaceTablet: tablet fd4021dcc28f40fdb505f2491e034a40 deleted and replaced by tablet 8a5b3f2a01b5453983b0d767ff376a6c
I20240503 02:09:16.063035 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b402f1be4d4fab80a0cef3392725bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.063474 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet fd4021dcc28f40fdb505f2491e034a40 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a5b3f2a01b5453983b0d767ff376a6c) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.063629 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.063706 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26dbf9f67602403e8d0b1b2fb82acebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.063733 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet fd4021dcc28f40fdb505f2491e034a40 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a5b3f2a01b5453983b0d767ff376a6c) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.064146 27562 tablet_service.cc:1467] Processing CreateTablet for tablet c9b402f1be4d4fab80a0cef3392725bf (DEFAULT_TABLE table=default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.064145 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_50cc9cf2680d4022a8e6a059b1ab6ad7" } new_table_name: "default.Table_3826661f099d46e0bacb75ec4454ee94" modify_external_catalogs: true
I20240503 02:09:16.064232 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b402f1be4d4fab80a0cef3392725bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.064544 28766 consensus_queue.cc:1035] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.064667 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb59f7012094fab9cb807407fbc7996 (DEFAULT_TABLE table=default.Table_a332ac50155641dc87ea83b511517566 [id=6d56982ce7944fd6860d7a722a553b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.064739 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fb59f7012094fab9cb807407fbc7996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.064782 28710 tablet.cc:1722] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.065402 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb59f7012094fab9cb807407fbc7996 (DEFAULT_TABLE table=default.Table_a332ac50155641dc87ea83b511517566 [id=6d56982ce7944fd6860d7a722a553b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.065402 27692 tablet_service.cc:1467] Processing CreateTablet for tablet c9b402f1be4d4fab80a0cef3392725bf (DEFAULT_TABLE table=default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.065498 27693 tablet_service.cc:1467] Processing CreateTablet for tablet 60368acf76cd4e9eb8e6750b37ad95b9 (DEFAULT_TABLE table=default.Table_f496c1bdb19a424a9ef4d1f772825869 [id=57d220ab05944f938c205ffe5861eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.065542 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.065551 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet fd4021dcc28f40fdb505f2491e034a40 with delete_type TABLET_DATA_DELETED (replaced by tablet 8a5b3f2a01b5453983b0d767ff376a6c) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.065598 27691 tablet_service.cc:1467] Processing CreateTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.065613 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26dbf9f67602403e8d0b1b2fb82acebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.065721 28654 tablet_replica.cc:330] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.065775 28654 raft_consensus.cc:2229] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.065811 28654 raft_consensus.cc:2258] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.065924 29090 mvcc.cc:205] Tried to move back new op lower bound from 7023420031207870464 to 7023420030850244608. Current Snapshot: MvccSnapshot[applied={T|T < 7023420031207870464}]
I20240503 02:09:16.066020 28654 ts_tablet_manager.cc:1875] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.066184 28774 raft_consensus.cc:482] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.066228 28774 raft_consensus.cc:504] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.066334 28774 leader_election.cc:288] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.066710 28653 tablet_replica.cc:330] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.066766 28653 raft_consensus.cc:2229] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.066818 28653 raft_consensus.cc:2258] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.066804 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9df0eb41d54a7ca550c0323c305424" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.066860 27583 raft_consensus.cc:2454] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.067039 27656 leader_election.cc:302] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.067111 28774 raft_consensus.cc:2781] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.067133 28774 raft_consensus.cc:482] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.067148 28774 raft_consensus.cc:3037] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.067143 28654 ts_tablet_manager.cc:1888] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.067180 28654 log.cc:1196] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fd4021dcc28f40fdb505f2491e034a40
I20240503 02:09:16.067230 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 60368acf76cd4e9eb8e6750b37ad95b9 (DEFAULT_TABLE table=default.Table_f496c1bdb19a424a9ef4d1f772825869 [id=57d220ab05944f938c205ffe5861eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.067313 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60368acf76cd4e9eb8e6750b37ad95b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.067353 28654 ts_tablet_manager.cc:1909] T fd4021dcc28f40fdb505f2491e034a40 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.067574 28706 tablet.cc:1722] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.067772 28774 raft_consensus.cc:504] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.067873 28774 leader_election.cc:288] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.068257 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:16.068336 29172 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.068375 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9df0eb41d54a7ca550c0323c305424" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.068425 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9df0eb41d54a7ca550c0323c305424" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.068447 27451 raft_consensus.cc:3037] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.068470 27582 raft_consensus.cc:3037] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.068576 28664 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.068612 28664 ts_tablet_manager.cc:1367] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:16.068737 28664 raft_consensus.cc:348] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.068784 28664 raft_consensus.cc:374] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.068802 28664 raft_consensus.cc:729] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.068830 28664 consensus_queue.cc:260] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.068908 28664 ts_tablet_manager.cc:1398] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.068957 28664 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.068977 27451 raft_consensus.cc:2454] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.068977 27582 raft_consensus.cc:2454] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.068377 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd9df0eb41d54a7ca550c0323c305424" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.069085 27449 raft_consensus.cc:2362] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:16.069269 28664 tablet_bootstrap.cc:654] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.069468 27656 leader_election.cc:302] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.069531 27853 raft_consensus.cc:2781] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.069561 27853 raft_consensus.cc:686] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.069591 27853 consensus_queue.cc:237] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.069643 28698 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.070222 28653 ts_tablet_manager.cc:1875] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.070367 28655 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.070406 28655 ts_tablet_manager.cc:1367] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.001s
I20240503 02:09:16.070535 28655 raft_consensus.cc:348] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.070587 28655 raft_consensus.cc:374] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.070605 28655 raft_consensus.cc:729] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.070633 28655 consensus_queue.cc:260] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.070715 28655 ts_tablet_manager.cc:1398] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.070770 28655 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.071103 28655 tablet_bootstrap.cc:654] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.071414 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.071513 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_7de06993e4e948b3bf97dad177b17e83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.071597 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7de06993e4e948b3bf97dad177b17e83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.071601 28653 ts_tablet_manager.cc:1888] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.071638 28653 log.cc:1196] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fd4021dcc28f40fdb505f2491e034a40
I20240503 02:09:16.071940 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ef979a5f65c4465987fed4ac2de01fb8" } new_table_name: "default.Table_981eecb56462490eaeee1cc6e0d2eaa5" modify_external_catalogs: true
I20240503 02:09:16.072006 28653 ts_tablet_manager.cc:1909] T fd4021dcc28f40fdb505f2491e034a40 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.073062 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.073146 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5b3f2a01b5453983b0d767ff376a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.073618 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:16.073935 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_63bcae89d666400790b5517d39e533ca [id=cd9e7541be36427ea8e01bbba147d869] alter complete (version 1)
I20240503 02:09:16.074194 28655 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.074232 28655 ts_tablet_manager.cc:1367] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.074246 28664 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.074280 28664 ts_tablet_manager.cc:1367] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.074391 28664 raft_consensus.cc:348] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.074438 28664 raft_consensus.cc:374] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.074455 28664 raft_consensus.cc:729] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.074419 28655 raft_consensus.cc:348] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.074481 28655 raft_consensus.cc:374] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.074498 28655 raft_consensus.cc:729] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.074510 28664 consensus_queue.cc:260] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.074529 28655 consensus_queue.cc:260] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.074539 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 138ffe8057904a2485289b61064bf382 (table default.Table_63bcae89d666400790b5517d39e533ca [id=cd9e7541be36427ea8e01bbba147d869]). Expected version 1 got 0
I20240503 02:09:16.074590 28664 ts_tablet_manager.cc:1398] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.074609 28655 ts_tablet_manager.cc:1398] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.074653 28664 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.074657 28655 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.074971 28664 tablet_bootstrap.cc:654] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.075002 28655 tablet_bootstrap.cc:654] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.075317 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_40eaf19db62e4a6a80b0322cc00fa0f3" } new_table_name: "default.Table_fe126d10909141ffaf718841bfe77ce2" modify_external_catalogs: true
I20240503 02:09:16.075431 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_3826661f099d46e0bacb75ec4454ee94 [id=87148f42b2134ac5accf54e9b542661a]). Expected version 2 got 1
I20240503 02:09:16.075546 28698 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.075788 27202 catalog_manager.cc:5484] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.076061 28710 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20240503 02:09:16.076120 27201 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fd9df0eb41d54a7ca550c0323c305424 consensus state has no leader
I20240503 02:09:16.076316 29167 tablet.cc:1722] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.077215 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.077314 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5b3f2a01b5453983b0d767ff376a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.078526 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81] alter complete (version 1)
I20240503 02:09:16.078547 28694 tablet.cc:1722] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.079223 28664 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.079267 28664 ts_tablet_manager.cc:1367] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:16.079389 28664 raft_consensus.cc:348] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.079440 28664 raft_consensus.cc:374] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.079460 28664 raft_consensus.cc:729] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.079434 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 60368acf76cd4e9eb8e6750b37ad95b9 (DEFAULT_TABLE table=default.Table_f496c1bdb19a424a9ef4d1f772825869 [id=57d220ab05944f938c205ffe5861eaa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.079526 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60368acf76cd4e9eb8e6750b37ad95b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.079510 28664 consensus_queue.cc:260] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.079617 28664 ts_tablet_manager.cc:1398] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.080576 27433 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb59f7012094fab9cb807407fbc7996 (DEFAULT_TABLE table=default.Table_a332ac50155641dc87ea83b511517566 [id=6d56982ce7944fd6860d7a722a553b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.080659 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fb59f7012094fab9cb807407fbc7996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.081655 27690 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce0b8f7896e400c8847800bccdda58f (DEFAULT_TABLE table=default.Table_7de06993e4e948b3bf97dad177b17e83 [id=6e100540683040cfa172ba132e64928d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.082609 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 3bc72a7c4997486280a2fdcb3bc07c81 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.082732 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce0b8f7896e400c8847800bccdda58f (DEFAULT_TABLE table=default.Table_7de06993e4e948b3bf97dad177b17e83 [id=6e100540683040cfa172ba132e64928d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.082813 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce0b8f7896e400c8847800bccdda58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.083127 28710 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.083215 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3826661f099d46e0bacb75ec4454ee94 [id=87148f42b2134ac5accf54e9b542661a] alter complete (version 2)
I20240503 02:09:16.083983 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce0b8f7896e400c8847800bccdda58f (DEFAULT_TABLE table=default.Table_7de06993e4e948b3bf97dad177b17e83 [id=6e100540683040cfa172ba132e64928d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.084069 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce0b8f7896e400c8847800bccdda58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.084544 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3bc72a7c4997486280a2fdcb3bc07c81
I20240503 02:09:16.084627 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 3bc72a7c4997486280a2fdcb3bc07c81 deleted and replaced by tablet f4a1100e58af4ce980576a3e78b98600
I20240503 02:09:16.084831 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 with delete_type TABLET_DATA_DELETED (replaced by tablet f4a1100e58af4ce980576a3e78b98600) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.085142 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 with delete_type TABLET_DATA_DELETED (replaced by tablet f4a1100e58af4ce980576a3e78b98600) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.085250 28654 tablet_replica.cc:330] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.085304 28654 raft_consensus.cc:2229] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.085424 28654 raft_consensus.cc:2258] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.085597 28654 ts_tablet_manager.cc:1875] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.085844 28655 tablet_bootstrap.cc:492] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.085898 28655 ts_tablet_manager.cc:1367] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:16.086036 28655 raft_consensus.cc:348] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.086086 28655 raft_consensus.cc:374] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.086103 28655 raft_consensus.cc:729] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.086140 28655 consensus_queue.cc:260] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.086220 28655 ts_tablet_manager.cc:1398] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.086278 28655 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.086550 28655 tablet_bootstrap.cc:654] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.086701 28654 ts_tablet_manager.cc:1888] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.086743 28654 log.cc:1196] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3bc72a7c4997486280a2fdcb3bc07c81
I20240503 02:09:16.086903 28654 ts_tablet_manager.cc:1909] T 3bc72a7c4997486280a2fdcb3bc07c81 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.086999 27702 tablet_service.cc:1467] Processing CreateTablet for tablet f4a1100e58af4ce980576a3e78b98600 (DEFAULT_TABLE table=default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.087030 28655 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.087069 28655 ts_tablet_manager.cc:1367] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.087136 27562 tablet_service.cc:1467] Processing CreateTablet for tablet f4a1100e58af4ce980576a3e78b98600 (DEFAULT_TABLE table=default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.087215 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a1100e58af4ce980576a3e78b98600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.087213 28655 raft_consensus.cc:348] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.087281 28655 raft_consensus.cc:374] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.087306 28655 raft_consensus.cc:729] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.087339 28655 consensus_queue.cc:260] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.087443 28655 ts_tablet_manager.cc:1398] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.087502 28655 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.087822 28655 tablet_bootstrap.cc:654] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.088580 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bc72a7c4997486280a2fdcb3bc07c81 with delete_type TABLET_DATA_DELETED (replaced by tablet f4a1100e58af4ce980576a3e78b98600) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.088627 27720 raft_consensus.cc:1261] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.088740 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3bc72a7c4997486280a2fdcb3bc07c81 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]) successfully deleted
I20240503 02:09:16.088344 28655 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.088809 28767 consensus_queue.cc:1035] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.088939 28655 ts_tablet_manager.cc:1367] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.089028 27587 raft_consensus.cc:1261] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.089244 28653 tablet_replica.cc:330] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.089313 28653 raft_consensus.cc:2229] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.089381 28653 raft_consensus.cc:2258] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.090087 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b402f1be4d4fab80a0cef3392725bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.089524 28653 ts_tablet_manager.cc:1875] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.090152 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce0b8f7896e400c8847800bccdda58f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.090207 27693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60368acf76cd4e9eb8e6750b37ad95b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.091341 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5b3f2a01b5453983b0d767ff376a6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.091367 28653 ts_tablet_manager.cc:1888] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.091409 28653 log.cc:1196] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3bc72a7c4997486280a2fdcb3bc07c81
I20240503 02:09:16.091470 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26dbf9f67602403e8d0b1b2fb82acebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.091521 28664 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.091688 28653 ts_tablet_manager.cc:1909] T 3bc72a7c4997486280a2fdcb3bc07c81 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.091945 28664 tablet_bootstrap.cc:654] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.092200 28655 raft_consensus.cc:348] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.092254 28655 raft_consensus.cc:374] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.092272 28655 raft_consensus.cc:729] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.092303 28655 consensus_queue.cc:260] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.092373 28664 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.092406 28664 ts_tablet_manager.cc:1367] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.092433 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a1100e58af4ce980576a3e78b98600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.092520 28664 raft_consensus.cc:348] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.092569 28664 raft_consensus.cc:374] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.092586 28664 raft_consensus.cc:729] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.092617 28664 consensus_queue.cc:260] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.092689 28664 ts_tablet_manager.cc:1398] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.092734 28664 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.093068 28664 tablet_bootstrap.cc:654] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.093197 28655 ts_tablet_manager.cc:1398] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.093475 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fb59f7012094fab9cb807407fbc7996. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.093758 28655 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.094110 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4bda2dc4192e4d2c87ff89620b509cd7" } new_table_name: "default.Table_2079721e8b764ff48a560052fa95adfa" modify_external_catalogs: true
I20240503 02:09:16.094280 28655 tablet_bootstrap.cc:654] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.089929 27426 tablet_service.cc:1467] Processing CreateTablet for tablet f4a1100e58af4ce980576a3e78b98600 (DEFAULT_TABLE table=default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.094662 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a1100e58af4ce980576a3e78b98600. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.094836 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3bc72a7c4997486280a2fdcb3bc07c81 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]) successfully deleted
I20240503 02:09:16.095340 28664 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.095384 28664 ts_tablet_manager.cc:1367] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.095515 28664 raft_consensus.cc:348] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.095572 28664 raft_consensus.cc:374] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.095589 28664 raft_consensus.cc:729] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.095619 28664 consensus_queue.cc:260] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.095701 28664 ts_tablet_manager.cc:1398] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.095760 28664 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.095942 28409 consensus_queue.cc:1035] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.095992 27719 raft_consensus.cc:1261] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.096073 28664 tablet_bootstrap.cc:654] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.096263 28409 consensus_queue.cc:1035] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.096313 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (table default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7]). Expected version 1 got 0
I20240503 02:09:16.096653 28661 ts_tablet_manager.cc:1888] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.096869 27587 raft_consensus.cc:1261] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.097699 28661 log.cc:1196] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/bdc8839c6cb14c648d15ef51ccb722ea
I20240503 02:09:16.097723 28655 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.097760 28655 ts_tablet_manager.cc:1367] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.097884 28655 raft_consensus.cc:348] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.097942 28655 raft_consensus.cc:374] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.097960 28655 raft_consensus.cc:729] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.097998 28661 ts_tablet_manager.cc:1909] T bdc8839c6cb14c648d15ef51ccb722ea P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.097993 28655 consensus_queue.cc:260] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.098081 28655 ts_tablet_manager.cc:1398] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.098134 28655 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.098309 28661 tablet_replica.cc:330] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.098307 28767 consensus_queue.cc:1035] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.098332 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet bdc8839c6cb14c648d15ef51ccb722ea (table default.Table_11b878bc4a1740f2ab615579a62a5cfd [id=30083f99eaa345d6bfedf2c59bfd376b]) successfully deleted
I20240503 02:09:16.098449 28655 tablet_bootstrap.cc:654] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.098899 28661 raft_consensus.cc:2229] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.098951 28661 raft_consensus.cc:2258] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.099022 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6f96d95848404e34a565d59b7e182cc5" } modify_external_catalogs: true
I20240503 02:09:16.099078 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6f96d95848404e34a565d59b7e182cc5" } modify_external_catalogs: true
I20240503 02:09:16.099215 28656 tablet_bootstrap.cc:492] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.099223 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 811e1aa52cb2458792359e9baebd5e5c (table default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8]). Expected version 1 got 0
I20240503 02:09:16.099254 28656 ts_tablet_manager.cc:1367] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20240503 02:09:16.099370 28656 raft_consensus.cc:348] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.099422 28656 raft_consensus.cc:374] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.099439 28656 raft_consensus.cc:729] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.099464 28656 consensus_queue.cc:260] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.099555 28656 ts_tablet_manager.cc:1398] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.099604 28656 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.100013 28656 tablet_bootstrap.cc:654] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.100486 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (table default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7]). Expected version 1 got 0
I20240503 02:09:16.101085 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 81d0a40d51684646a0c7964f2a265df9 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.101114 28661 ts_tablet_manager.cc:1875] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.101172 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 811e1aa52cb2458792359e9baebd5e5c (table default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8]). Expected version 1 got 0
I20240503 02:09:16.101297 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_15d9a598c1ea470c8376e98dfea36c42" } modify_external_catalogs: true
I20240503 02:09:16.101363 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_15d9a598c1ea470c8376e98dfea36c42" } modify_external_catalogs: true
I20240503 02:09:16.101672 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 02a45a0a0f0a44099a10ec036aaf3acf from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.102321 28661 ts_tablet_manager.cc:1888] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.102417 28661 log.cc:1196] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fd4021dcc28f40fdb505f2491e034a40
I20240503 02:09:16.102617 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 0c35a98c21b848e79a38f1b9cf5919a6
I20240503 02:09:16.102649 28661 ts_tablet_manager.cc:1909] T fd4021dcc28f40fdb505f2491e034a40 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.102682 28767 consensus_queue.cc:1035] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.102835 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.102880 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.102949 28654 tablet_replica.cc:330] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.102991 28654 raft_consensus.cc:2229] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.103031 28654 pending_rounds.cc:62] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:16.103052 28654 pending_rounds.cc:68] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420030960689152 op_type: NO_OP noop_request { }
I20240503 02:09:16.103083 28654 raft_consensus.cc:2866] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.103102 28654 raft_consensus.cc:2258] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.103482 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 81d0a40d51684646a0c7964f2a265df9
I20240503 02:09:16.103565 27202 catalog_manager.cc:6554] ReplaceTablet: tablet 81d0a40d51684646a0c7964f2a265df9 deleted and replaced by tablet 322fe14cc0844c2b842ba27e21451535
I20240503 02:09:16.103587 28654 ts_tablet_manager.cc:1875] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.103957 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4e4b1db49a3d4c1085b24c95509fe6a6
I20240503 02:09:16.104202 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.104313 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 02a45a0a0f0a44099a10ec036aaf3acf
I20240503 02:09:16.104393 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 02a45a0a0f0a44099a10ec036aaf3acf deleted and replaced by tablet e776dc8757154957900b9fdecbe4dcc1
I20240503 02:09:16.104535 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf with delete_type TABLET_DATA_DELETED (replaced by tablet e776dc8757154957900b9fdecbe4dcc1) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.104714 28654 ts_tablet_manager.cc:1888] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.104756 28654 log.cc:1196] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0c35a98c21b848e79a38f1b9cf5919a6
I20240503 02:09:16.104782 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d0a40d51684646a0c7964f2a265df9 with delete_type TABLET_DATA_DELETED (replaced by tablet 322fe14cc0844c2b842ba27e21451535) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.104848 28653 tablet_replica.cc:330] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.104856 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf with delete_type TABLET_DATA_DELETED (replaced by tablet e776dc8757154957900b9fdecbe4dcc1) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.104905 28653 raft_consensus.cc:2229] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.104935 28653 raft_consensus.cc:2258] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.104943 28654 ts_tablet_manager.cc:1909] T 0c35a98c21b848e79a38f1b9cf5919a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.105141 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d0a40d51684646a0c7964f2a265df9 with delete_type TABLET_DATA_DELETED (replaced by tablet 322fe14cc0844c2b842ba27e21451535) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.105176 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.105252 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a45a0a0f0a44099a10ec036aaf3acf with delete_type TABLET_DATA_DELETED (replaced by tablet e776dc8757154957900b9fdecbe4dcc1) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.105268 28653 ts_tablet_manager.cc:1875] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.105371 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e4b1db49a3d4c1085b24c95509fe6a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.105433 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 81d0a40d51684646a0c7964f2a265df9 with delete_type TABLET_DATA_DELETED (replaced by tablet 322fe14cc0844c2b842ba27e21451535) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.105535 28654 tablet_replica.cc:330] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.105573 28654 raft_consensus.cc:2229] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.105602 28654 raft_consensus.cc:2258] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.105739 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0c35a98c21b848e79a38f1b9cf5919a6 (table default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]) successfully deleted
I20240503 02:09:16.105775 27450 raft_consensus.cc:1261] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.105966 27853 consensus_queue.cc:1035] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.106099 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c35a98c21b848e79a38f1b9cf5919a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.106247 28661 tablet_replica.cc:330] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.106295 28661 raft_consensus.cc:2229] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.106325 28661 raft_consensus.cc:2258] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.106336 27585 raft_consensus.cc:1261] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.106376 28653 ts_tablet_manager.cc:1888] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.106411 28653 log.cc:1196] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/81d0a40d51684646a0c7964f2a265df9
I20240503 02:09:16.106500 27853 consensus_queue.cc:1035] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.106576 28653 ts_tablet_manager.cc:1909] T 81d0a40d51684646a0c7964f2a265df9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.106850 28653 tablet_replica.cc:330] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.106845 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 9a8e43c8d5554686aa9ecee00d265c92 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.106909 28653 raft_consensus.cc:2229] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.106938 28653 raft_consensus.cc:2258] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.107015 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fd4021dcc28f40fdb505f2491e034a40 (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:16.107023 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe14cc0844c2b842ba27e21451535 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.107098 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe14cc0844c2b842ba27e21451535. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.107167 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 81d0a40d51684646a0c7964f2a265df9 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.108207 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe14cc0844c2b842ba27e21451535 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.108292 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe14cc0844c2b842ba27e21451535. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.108512 28654 ts_tablet_manager.cc:1875] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.108911 28653 ts_tablet_manager.cc:1875] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.109295 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9a8e43c8d5554686aa9ecee00d265c92
I20240503 02:09:16.109395 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 9a8e43c8d5554686aa9ecee00d265c92 deleted and replaced by tablet d8424c39f615456e8db186a9c124629c
I20240503 02:09:16.109511 27702 tablet_service.cc:1467] Processing CreateTablet for tablet e776dc8757154957900b9fdecbe4dcc1 (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.109583 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e776dc8757154957900b9fdecbe4dcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.110157 28654 ts_tablet_manager.cc:1888] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.110195 28654 log.cc:1196] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4e4b1db49a3d4c1085b24c95509fe6a6
I20240503 02:09:16.110373 28654 ts_tablet_manager.cc:1909] T 4e4b1db49a3d4c1085b24c95509fe6a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.110631 28653 ts_tablet_manager.cc:1888] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.110671 28653 log.cc:1196] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/02a45a0a0f0a44099a10ec036aaf3acf
I20240503 02:09:16.110687 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:16.110838 28653 ts_tablet_manager.cc:1909] T 02a45a0a0f0a44099a10ec036aaf3acf P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.111016 28653 tablet_replica.cc:330] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.111060 28653 raft_consensus.cc:2229] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.111078 29046 mvcc.cc:205] Tried to move back new op lower bound from 7023420031407755264 to 7023420031261167616. Current Snapshot: MvccSnapshot[applied={T|T < 7023420031407755264}]
I20240503 02:09:16.111109 28653 raft_consensus.cc:2258] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.111162 29167 tablet.cc:1722] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.111160 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 02a45a0a0f0a44099a10ec036aaf3acf (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:16.111169 28694 tablet.cc:1722] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.111271 27562 tablet_service.cc:1467] Processing CreateTablet for tablet e776dc8757154957900b9fdecbe4dcc1 (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.111346 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 322fe14cc0844c2b842ba27e21451535 (DEFAULT_TABLE table=default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.111455 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 with delete_type TABLET_DATA_DELETED (replaced by tablet d8424c39f615456e8db186a9c124629c) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.111550 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 with delete_type TABLET_DATA_DELETED (replaced by tablet d8424c39f615456e8db186a9c124629c) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.112119 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 322fe14cc0844c2b842ba27e21451535. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.112510 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:16.113101 27427 tablet_service.cc:1467] Processing CreateTablet for tablet e776dc8757154957900b9fdecbe4dcc1 (DEFAULT_TABLE table=default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.113178 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e776dc8757154957900b9fdecbe4dcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.113350 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:16.114007 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a8e43c8d5554686aa9ecee00d265c92 with delete_type TABLET_DATA_DELETED (replaced by tablet d8424c39f615456e8db186a9c124629c) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.114245 28653 ts_tablet_manager.cc:1875] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.114359 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d8424c39f615456e8db186a9c124629c (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.114423 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d8424c39f615456e8db186a9c124629c (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.114502 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8424c39f615456e8db186a9c124629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.114665 27703 tablet_service.cc:1467] Processing CreateTablet for tablet d8424c39f615456e8db186a9c124629c (DEFAULT_TABLE table=default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.114745 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8424c39f615456e8db186a9c124629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.115675 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e776dc8757154957900b9fdecbe4dcc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.116407 28653 ts_tablet_manager.cc:1888] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.116448 28653 log.cc:1196] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4e4b1db49a3d4c1085b24c95509fe6a6
I20240503 02:09:16.116631 28653 ts_tablet_manager.cc:1909] T 4e4b1db49a3d4c1085b24c95509fe6a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.116870 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8424c39f615456e8db186a9c124629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.116948 28653 tablet_replica.cc:330] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.117004 28653 raft_consensus.cc:2229] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.117033 28653 raft_consensus.cc:2258] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.117098 27201 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0] alter complete (version 1)
I20240503 02:09:16.117269 28661 ts_tablet_manager.cc:1875] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.117370 28664 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.117419 28664 ts_tablet_manager.cc:1367] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20240503 02:09:16.117548 28664 raft_consensus.cc:348] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.117612 28664 raft_consensus.cc:374] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.117636 28664 raft_consensus.cc:729] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.117671 28664 consensus_queue.cc:260] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.117849 28664 ts_tablet_manager.cc:1398] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.117906 28664 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.118467 28664 tablet_bootstrap.cc:654] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.118626 28710 tablet.cc:1722] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.118891 28661 ts_tablet_manager.cc:1888] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.118934 28661 log.cc:1196] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3bc72a7c4997486280a2fdcb3bc07c81
I20240503 02:09:16.119108 28661 ts_tablet_manager.cc:1909] T 3bc72a7c4997486280a2fdcb3bc07c81 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.119299 28661 tablet_replica.cc:330] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.119345 28661 raft_consensus.cc:2229] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.119342 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (table default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]) successfully deleted
I20240503 02:09:16.119374 28661 raft_consensus.cc:2258] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.119371 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3bc72a7c4997486280a2fdcb3bc07c81 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]) successfully deleted
I20240503 02:09:16.119483 28653 ts_tablet_manager.cc:1875] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.120141 28661 ts_tablet_manager.cc:1875] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.120522 29167 tablet.cc:1722] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.120652 28410 raft_consensus.cc:482] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.120695 28410 raft_consensus.cc:504] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.120783 28410 leader_election.cc:288] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.120920 28653 ts_tablet_manager.cc:1888] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.120889 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78330603666e4171845cf14dade17795" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.120960 28653 log.cc:1196] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0c35a98c21b848e79a38f1b9cf5919a6
I20240503 02:09:16.120965 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78330603666e4171845cf14dade17795" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.121018 27720 raft_consensus.cc:2454] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.120965 27453 raft_consensus.cc:2454] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.121120 28653 ts_tablet_manager.cc:1909] T 0c35a98c21b848e79a38f1b9cf5919a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.121145 27523 leader_election.cc:302] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.121212 28410 raft_consensus.cc:2781] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.121235 28410 raft_consensus.cc:482] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.121250 28410 raft_consensus.cc:3037] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.121315 28653 tablet_replica.cc:330] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.121374 28653 raft_consensus.cc:2229] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.121404 28653 raft_consensus.cc:2258] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.121595 28653 ts_tablet_manager.cc:1875] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.121616 28661 ts_tablet_manager.cc:1888] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.121609 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0c35a98c21b848e79a38f1b9cf5919a6 (table default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]) successfully deleted
I20240503 02:09:16.121655 28661 log.cc:1196] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0c35a98c21b848e79a38f1b9cf5919a6
I20240503 02:09:16.121800 28661 ts_tablet_manager.cc:1909] T 0c35a98c21b848e79a38f1b9cf5919a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.121863 28410 raft_consensus.cc:504] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.121973 28410 leader_election.cc:288] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.122083 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78330603666e4171845cf14dade17795" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.122119 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0c35a98c21b848e79a38f1b9cf5919a6 (table default.Table_6f96d95848404e34a565d59b7e182cc5 [id=aa81b300b4a84844aa6cc4579a5bf31a]) successfully deleted
I20240503 02:09:16.122138 27453 raft_consensus.cc:3037] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.122129 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78330603666e4171845cf14dade17795" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.122180 27720 raft_consensus.cc:3037] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.122331 28710 tablet.cc:1722] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.122704 27453 raft_consensus.cc:2454] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.122717 27720 raft_consensus.cc:2454] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.122180 28661 tablet_replica.cc:330] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.122812 28661 raft_consensus.cc:2229] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.122838 28661 raft_consensus.cc:2258] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.122879 28655 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.122926 28655 ts_tablet_manager.cc:1367] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.000s
I20240503 02:09:16.122933 27524 leader_election.cc:302] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.123039 28661 ts_tablet_manager.cc:1875] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.123046 28655 raft_consensus.cc:348] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.123134 28655 raft_consensus.cc:374] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.123155 28655 raft_consensus.cc:729] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.123189 28655 consensus_queue.cc:260] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.122704 28653 ts_tablet_manager.cc:1888] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.123263 28655 ts_tablet_manager.cc:1398] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.123309 28655 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.123364 28507 raft_consensus.cc:2781] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.123405 28507 raft_consensus.cc:686] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.123435 28507 consensus_queue.cc:237] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.123620 28655 tablet_bootstrap.cc:654] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.123826 28653 log.cc:1196] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9a8e43c8d5554686aa9ecee00d265c92
I20240503 02:09:16.123049 28685 raft_consensus.cc:482] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.123924 28685 raft_consensus.cc:504] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.123994 28685 leader_election.cc:288] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.124131 28653 ts_tablet_manager.cc:1909] T 9a8e43c8d5554686aa9ecee00d265c92 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.124186 28685 consensus_queue.cc:1035] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.124301 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23129af1b17c45e093e85c2b0032f864" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.124352 27723 raft_consensus.cc:2454] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.124428 28694 tablet.cc:1722] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.124450 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9a8e43c8d5554686aa9ecee00d265c92 (table default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]) successfully deleted
I20240503 02:09:16.124514 28656 tablet_bootstrap.cc:492] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.124549 28656 ts_tablet_manager.cc:1367] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240503 02:09:16.124663 28656 raft_consensus.cc:348] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.124706 28656 raft_consensus.cc:374] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.124722 28656 raft_consensus.cc:729] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.124754 28656 consensus_queue.cc:260] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.124825 28656 ts_tablet_manager.cc:1398] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.124905 27388 leader_election.cc:302] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.125033 28685 raft_consensus.cc:2781] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.125061 28685 raft_consensus.cc:482] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.125077 28685 raft_consensus.cc:3037] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.125802 28685 raft_consensus.cc:504] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.125828 28664 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.125869 28664 ts_tablet_manager.cc:1367] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:16.125913 28685 leader_election.cc:288] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.126009 28664 raft_consensus.cc:348] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.126235 28664 raft_consensus.cc:374] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.126263 28664 raft_consensus.cc:729] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.126307 28655 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.126343 28655 ts_tablet_manager.cc:1367] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.126454 28655 raft_consensus.cc:348] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.126518 28655 raft_consensus.cc:374] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.126536 28655 raft_consensus.cc:729] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.126305 28664 consensus_queue.cc:260] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.126266 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23129af1b17c45e093e85c2b0032f864" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.126566 28655 consensus_queue.cc:260] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.126603 27723 raft_consensus.cc:3037] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.126634 28664 ts_tablet_manager.cc:1398] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.126642 28655 ts_tablet_manager.cc:1398] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.126688 28664 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.126694 28655 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.127038 28664 tablet_bootstrap.cc:654] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.127115 28655 tablet_bootstrap.cc:654] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.127210 27723 raft_consensus.cc:2454] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.127326 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7] alter complete (version 1)
I20240503 02:09:16.127750 27388 leader_election.cc:302] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.127815 28685 raft_consensus.cc:2781] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.127849 28685 raft_consensus.cc:686] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.127882 28685 consensus_queue.cc:237] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.127993 27201 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8] alter complete (version 1)
I20240503 02:09:16.128443 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
W20240503 02:09:16.128517 27201 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8a5b3f2a01b5453983b0d767ff376a6c consensus state has no leader
I20240503 02:09:16.129148 28694 tablet.cc:1722] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.129518 28661 ts_tablet_manager.cc:1888] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.129559 28661 log.cc:1196] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/81d0a40d51684646a0c7964f2a265df9
I20240503 02:09:16.129554 27201 catalog_manager.cc:5484] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.129634 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.129666 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:16.129726 28661 ts_tablet_manager.cc:1909] T 81d0a40d51684646a0c7964f2a265df9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.130283 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 81d0a40d51684646a0c7964f2a265df9 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.130357 28661 tablet_replica.cc:330] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.130399 28661 raft_consensus.cc:2229] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.130419 28661 raft_consensus.cc:2258] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.130584 28661 ts_tablet_manager.cc:1875] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.130806 28710 tablet.cc:1722] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.131798 29241 raft_consensus.cc:482] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.131841 29241 raft_consensus.cc:504] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.131951 29241 leader_election.cc:288] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.132079 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06be437ed5114166b802559170e440eb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.132138 27450 raft_consensus.cc:2454] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.132469 27524 leader_election.cc:302] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.132452 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06be437ed5114166b802559170e440eb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.132529 27720 raft_consensus.cc:2454] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.132547 28661 ts_tablet_manager.cc:1888] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.132587 28661 log.cc:1196] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4e4b1db49a3d4c1085b24c95509fe6a6
I20240503 02:09:16.132608 29241 raft_consensus.cc:2781] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.132642 29241 raft_consensus.cc:482] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.132661 29241 raft_consensus.cc:3037] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.132956 28661 ts_tablet_manager.cc:1909] T 4e4b1db49a3d4c1085b24c95509fe6a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.133234 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (table default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]) successfully deleted
I20240503 02:09:16.133229 29241 raft_consensus.cc:504] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.133350 28661 tablet_replica.cc:330] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.133457 28661 raft_consensus.cc:2229] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.133534 28661 raft_consensus.cc:2258] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.133741 29241 leader_election.cc:288] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.133854 28661 ts_tablet_manager.cc:1875] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.133991 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06be437ed5114166b802559170e440eb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.134043 27450 raft_consensus.cc:3037] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.134088 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06be437ed5114166b802559170e440eb" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.134142 27720 raft_consensus.cc:3037] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.134591 27450 raft_consensus.cc:2454] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.134582 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23129af1b17c45e093e85c2b0032f864" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.134642 27583 raft_consensus.cc:2454] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.134699 27720 raft_consensus.cc:2454] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.134778 27719 raft_consensus.cc:1261] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.134794 29167 tablet.cc:1722] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.134992 27585 raft_consensus.cc:3037] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.135318 28740 consensus_queue.cc:1035] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.135594 27585 raft_consensus.cc:1261] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.135774 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23129af1b17c45e093e85c2b0032f864" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.136132 28661 ts_tablet_manager.cc:1888] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.136173 28661 log.cc:1196] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/02a45a0a0f0a44099a10ec036aaf3acf
I20240503 02:09:16.136308 27582 raft_consensus.cc:2454] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.136499 27524 leader_election.cc:302] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.136682 28766 consensus_queue.cc:1035] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.136859 28661 ts_tablet_manager.cc:1909] T 02a45a0a0f0a44099a10ec036aaf3acf P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.137059 28507 raft_consensus.cc:2781] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.137110 28507 raft_consensus.cc:686] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.137142 28507 consensus_queue.cc:237] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.137408 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 02a45a0a0f0a44099a10ec036aaf3acf (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:16.137635 28661 tablet_replica.cc:330] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.138067 28409 consensus_queue.cc:1035] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.138118 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.137977 27853 consensus_queue.cc:1035] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.138161 27201 catalog_manager.cc:5484] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.138231 27201 catalog_manager.cc:5484] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.137691 28661 raft_consensus.cc:2229] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.138310 28661 raft_consensus.cc:2258] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.138494 28661 ts_tablet_manager.cc:1875] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.140698 28661 ts_tablet_manager.cc:1888] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.140790 28661 log.cc:1196] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9a8e43c8d5554686aa9ecee00d265c92
I20240503 02:09:16.140798 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.140973 28654 tablet_replica.cc:330] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.141024 28654 raft_consensus.cc:2229] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.141028 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4e4b1db49a3d4c1085b24c95509fe6a6 (table default.Table_15d9a598c1ea470c8376e98dfea36c42 [id=791b184cd1a14747a5066b11ede83f83]) successfully deleted
I20240503 02:09:16.141085 28654 raft_consensus.cc:2258] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.141188 28656 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.141310 28661 ts_tablet_manager.cc:1909] T 9a8e43c8d5554686aa9ecee00d265c92 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.141547 28656 tablet_bootstrap.cc:654] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.141559 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9a8e43c8d5554686aa9ecee00d265c92 (table default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]) successfully deleted
I20240503 02:09:16.141840 28655 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.141888 28655 ts_tablet_manager.cc:1367] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20240503 02:09:16.141891 28654 ts_tablet_manager.cc:1875] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.142022 28655 raft_consensus.cc:348] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.142076 28655 raft_consensus.cc:374] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.142094 28655 raft_consensus.cc:729] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.142123 28656 tablet_bootstrap.cc:492] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.142128 28655 consensus_queue.cc:260] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.142156 28656 ts_tablet_manager.cc:1367] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.142212 28655 ts_tablet_manager.cc:1398] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.142257 28655 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.142269 28656 raft_consensus.cc:348] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.142316 28656 raft_consensus.cc:374] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.142334 28656 raft_consensus.cc:729] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.142374 28656 consensus_queue.cc:260] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.142452 28656 ts_tablet_manager.cc:1398] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.142495 28656 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.142571 28655 tablet_bootstrap.cc:654] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.142802 28656 tablet_bootstrap.cc:654] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.143008 28654 ts_tablet_manager.cc:1888] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.143039 28664 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.143049 28654 log.cc:1196] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/02a45a0a0f0a44099a10ec036aaf3acf
I20240503 02:09:16.143072 28664 ts_tablet_manager.cc:1367] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240503 02:09:16.143191 28664 raft_consensus.cc:348] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.143246 28664 raft_consensus.cc:374] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.143250 28654 ts_tablet_manager.cc:1909] T 02a45a0a0f0a44099a10ec036aaf3acf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.143265 28664 raft_consensus.cc:729] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.143294 28664 consensus_queue.cc:260] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.143390 28664 ts_tablet_manager.cc:1398] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.143443 28664 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.143749 28664 tablet_bootstrap.cc:654] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.144174 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
W20240503 02:09:16.144243 27201 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f4a1100e58af4ce980576a3e78b98600 consensus state has no leader
I20240503 02:09:16.144399 28710 tablet.cc:1722] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.144603 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 02a45a0a0f0a44099a10ec036aaf3acf (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:16.144743 28654 tablet_replica.cc:330] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.144798 28654 raft_consensus.cc:2229] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.144850 28654 raft_consensus.cc:2258] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.145154 28654 ts_tablet_manager.cc:1875] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.145495 29167 tablet.cc:1722] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.145602 28656 tablet_bootstrap.cc:492] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.145646 28656 ts_tablet_manager.cc:1367] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.145762 28656 raft_consensus.cc:348] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.145812 28656 raft_consensus.cc:374] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.145828 28656 raft_consensus.cc:729] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.145849 28656 consensus_queue.cc:260] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.145932 28656 ts_tablet_manager.cc:1398] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.145983 28656 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.146230 28694 tablet.cc:1722] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.146301 28656 tablet_bootstrap.cc:654] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.147578 28654 ts_tablet_manager.cc:1888] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.147959 28655 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.148011 28655 ts_tablet_manager.cc:1367] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:16.148144 28655 raft_consensus.cc:348] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.148195 28655 raft_consensus.cc:374] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.148211 28655 raft_consensus.cc:729] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.148244 28655 consensus_queue.cc:260] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.148317 28655 ts_tablet_manager.cc:1398] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.148363 28655 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.148700 28655 tablet_bootstrap.cc:654] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.148823 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.148865 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.149032 28654 log.cc:1196] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/81d0a40d51684646a0c7964f2a265df9
I20240503 02:09:16.149255 28654 ts_tablet_manager.cc:1909] T 81d0a40d51684646a0c7964f2a265df9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.149583 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 81d0a40d51684646a0c7964f2a265df9 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.150215 28654 tablet_replica.cc:330] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.150306 28654 raft_consensus.cc:2229] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.150316 28655 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.150348 28655 ts_tablet_manager.cc:1367] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.150449 28654 raft_consensus.cc:2258] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.150480 28655 raft_consensus.cc:348] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.150533 28655 raft_consensus.cc:374] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.150552 28655 raft_consensus.cc:729] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.150588 28655 consensus_queue.cc:260] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.150662 28655 ts_tablet_manager.cc:1398] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.150604 28656 tablet_bootstrap.cc:492] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.150719 28655 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.150727 28656 ts_tablet_manager.cc:1367] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.150854 28656 raft_consensus.cc:348] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.150913 28656 raft_consensus.cc:374] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.150930 28656 raft_consensus.cc:729] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.150969 28656 consensus_queue.cc:260] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.151039 28655 tablet_bootstrap.cc:654] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.151062 28656 ts_tablet_manager.cc:1398] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.151116 28656 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.151412 28656 tablet_bootstrap.cc:654] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.151571 28654 ts_tablet_manager.cc:1875] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.152093 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.152238 28664 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.152287 28664 ts_tablet_manager.cc:1367] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:16.152426 28664 raft_consensus.cc:348] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.152487 28664 raft_consensus.cc:374] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.152503 28664 raft_consensus.cc:729] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.152535 28664 consensus_queue.cc:260] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.152609 28664 ts_tablet_manager.cc:1398] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.152681 28664 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.152992 28664 tablet_bootstrap.cc:654] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.153277 28656 tablet_bootstrap.cc:492] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.153457 28664 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.153455 28656 ts_tablet_manager.cc:1367] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.153508 28664 ts_tablet_manager.cc:1367] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.153589 28656 raft_consensus.cc:348] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.153646 28656 raft_consensus.cc:374] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.153630 28664 raft_consensus.cc:348] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.153669 28656 raft_consensus.cc:729] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.153676 28664 raft_consensus.cc:374] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.153692 28664 raft_consensus.cc:729] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.153705 28656 consensus_queue.cc:260] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.153728 28664 consensus_queue.cc:260] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.153793 28656 ts_tablet_manager.cc:1398] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.153793 28664 ts_tablet_manager.cc:1398] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.153867 28656 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.154196 28656 tablet_bootstrap.cc:654] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.154458 28654 ts_tablet_manager.cc:1888] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.154572 28664 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.154927 28654 log.cc:1196] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9a8e43c8d5554686aa9ecee00d265c92
I20240503 02:09:16.155030 28664 tablet_bootstrap.cc:654] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.155122 28654 ts_tablet_manager.cc:1909] T 9a8e43c8d5554686aa9ecee00d265c92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.155368 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9a8e43c8d5554686aa9ecee00d265c92 (table default.Table_d2e3a3ddf4ec4fc7b95fad1daf25dbf0 [id=a845816157b4481f996160b3912c69c1]) successfully deleted
I20240503 02:09:16.155668 28655 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.155710 28655 ts_tablet_manager.cc:1367] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:16.155740 28656 tablet_bootstrap.cc:492] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.155777 28656 ts_tablet_manager.cc:1367] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.155822 28655 raft_consensus.cc:348] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.155879 28655 raft_consensus.cc:374] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.155895 28655 raft_consensus.cc:729] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.155896 28656 raft_consensus.cc:348] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.155928 28655 consensus_queue.cc:260] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.155958 28656 raft_consensus.cc:374] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.155978 28656 raft_consensus.cc:729] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.155998 28655 ts_tablet_manager.cc:1398] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.156020 28656 consensus_queue.cc:260] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.156054 28655 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.156106 28656 ts_tablet_manager.cc:1398] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.156138 28664 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.156159 28656 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.156170 28664 ts_tablet_manager.cc:1367] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.156302 28664 raft_consensus.cc:348] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.156348 28664 raft_consensus.cc:374] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.156365 28664 raft_consensus.cc:729] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.156373 28655 tablet_bootstrap.cc:654] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.156399 28664 consensus_queue.cc:260] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.156473 28664 ts_tablet_manager.cc:1398] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.156508 28656 tablet_bootstrap.cc:654] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.156553 28664 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.156826 28655 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.156873 28664 tablet_bootstrap.cc:654] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.156867 28655 ts_tablet_manager.cc:1367] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.156993 28655 raft_consensus.cc:348] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.157039 28655 raft_consensus.cc:374] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.157056 28655 raft_consensus.cc:729] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.157083 28655 consensus_queue.cc:260] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.157167 28655 ts_tablet_manager.cc:1398] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.157392 28766 raft_consensus.cc:482] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.157444 28766 raft_consensus.cc:504] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.157539 28766 leader_election.cc:288] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.157716 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791439779c5b4f5f922403340c4b7a6f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.157780 27582 raft_consensus.cc:2454] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.158082 27386 leader_election.cc:302] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.158174 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791439779c5b4f5f922403340c4b7a6f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.158228 27720 raft_consensus.cc:2454] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.158454 28656 tablet_bootstrap.cc:492] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.158497 28656 ts_tablet_manager.cc:1367] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.158510 28655 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.158604 28656 raft_consensus.cc:348] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.158654 28656 raft_consensus.cc:374] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.158670 28656 raft_consensus.cc:729] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.158704 28656 consensus_queue.cc:260] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.158776 28656 ts_tablet_manager.cc:1398] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.158818 28656 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.158845 28655 tablet_bootstrap.cc:654] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.159216 28656 tablet_bootstrap.cc:654] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.159377 28766 raft_consensus.cc:2781] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.159430 28766 raft_consensus.cc:482] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.159447 28766 raft_consensus.cc:3037] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.160048 28766 raft_consensus.cc:504] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.160149 28766 leader_election.cc:288] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.160424 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791439779c5b4f5f922403340c4b7a6f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.160480 27582 raft_consensus.cc:3037] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.161012 27582 raft_consensus.cc:2454] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.161149 29241 raft_consensus.cc:482] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.161150 27386 leader_election.cc:302] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.161183 29241 raft_consensus.cc:504] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.161218 28766 raft_consensus.cc:2781] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.161257 28766 raft_consensus.cc:686] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.161273 29241 leader_election.cc:288] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:16.161295 28766 consensus_queue.cc:237] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.161363 28656 tablet_bootstrap.cc:492] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.161394 28656 ts_tablet_manager.cc:1367] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.161504 28656 raft_consensus.cc:348] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.161552 28656 raft_consensus.cc:374] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.161578 28656 raft_consensus.cc:729] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.161619 28656 consensus_queue.cc:260] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.161717 28656 ts_tablet_manager.cc:1398] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.161702 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce779dcce25548eba2fc39908daf2b9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.161763 27720 raft_consensus.cc:2454] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.161778 28656 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.161711 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "791439779c5b4f5f922403340c4b7a6f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.161958 27719 raft_consensus.cc:3037] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.162117 28656 tablet_bootstrap.cc:654] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.162537 27719 raft_consensus.cc:2454] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.162874 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce779dcce25548eba2fc39908daf2b9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.162935 27450 raft_consensus.cc:2454] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.163079 27524 leader_election.cc:302] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.163143 29241 raft_consensus.cc:2781] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.163165 29241 raft_consensus.cc:482] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.163182 29241 raft_consensus.cc:3037] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.163141 27196 catalog_manager.cc:5484] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.163736 29241 raft_consensus.cc:504] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.163828 29241 leader_election.cc:288] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:16.164254 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:16.164526 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce779dcce25548eba2fc39908daf2b9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.164528 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce779dcce25548eba2fc39908daf2b9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.164587 27450 raft_consensus.cc:3037] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.164587 27719 raft_consensus.cc:3037] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.164866 28664 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.164902 28664 ts_tablet_manager.cc:1367] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.164994 27719 raft_consensus.cc:2454] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.165133 27450 raft_consensus.cc:2454] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.165138 27523 leader_election.cc:302] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.165205 29241 raft_consensus.cc:2781] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.165055 28664 raft_consensus.cc:348] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.165235 29241 raft_consensus.cc:686] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.165247 28664 raft_consensus.cc:374] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.165273 28664 raft_consensus.cc:729] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.165266 29241 consensus_queue.cc:237] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.165310 28664 consensus_queue.cc:260] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.166103 27196 catalog_manager.cc:5484] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.166428 28664 ts_tablet_manager.cc:1398] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.167055 28664 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.167394 28664 tablet_bootstrap.cc:654] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.167901 28656 tablet_bootstrap.cc:492] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.167944 28656 ts_tablet_manager.cc:1367] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:16.168049 28656 raft_consensus.cc:348] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.168097 28656 raft_consensus.cc:374] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.168115 28656 raft_consensus.cc:729] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.168133 28664 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.168150 28656 consensus_queue.cc:260] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.168174 28664 ts_tablet_manager.cc:1367] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.168234 28656 ts_tablet_manager.cc:1398] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.168308 28656 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.168303 28664 raft_consensus.cc:348] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.168352 28664 raft_consensus.cc:374] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.168368 28664 raft_consensus.cc:729] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.168411 28664 consensus_queue.cc:260] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.168483 28664 ts_tablet_manager.cc:1398] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.168620 28656 tablet_bootstrap.cc:654] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.169203 28656 tablet_bootstrap.cc:492] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.169245 28656 ts_tablet_manager.cc:1367] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.169400 28656 raft_consensus.cc:348] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.169452 28656 raft_consensus.cc:374] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.169469 28656 raft_consensus.cc:729] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.169524 28656 consensus_queue.cc:260] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.169615 28656 ts_tablet_manager.cc:1398] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:16.169667 28656 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.169971 28656 tablet_bootstrap.cc:654] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.170428 28655 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.170472 28655 ts_tablet_manager.cc:1367] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:16.170580 28655 raft_consensus.cc:348] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.170634 28655 raft_consensus.cc:374] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.170653 28655 raft_consensus.cc:729] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.170683 28655 consensus_queue.cc:260] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.170779 28655 ts_tablet_manager.cc:1398] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.170893 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.171496 28656 tablet_bootstrap.cc:492] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.171545 28656 ts_tablet_manager.cc:1367] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.171659 28656 raft_consensus.cc:348] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.171710 28656 raft_consensus.cc:374] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.171727 28656 raft_consensus.cc:729] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.171761 28656 consensus_queue.cc:260] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.171844 28656 ts_tablet_manager.cc:1398] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.172206 28656 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.172554 28656 tablet_bootstrap.cc:654] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.172981 29241 raft_consensus.cc:482] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.173022 29241 raft_consensus.cc:504] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.173099 29241 leader_election.cc:288] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.173261 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26dbf9f67602403e8d0b1b2fb82acebc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.173267 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26dbf9f67602403e8d0b1b2fb82acebc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.173313 27719 raft_consensus.cc:2454] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.173358 27450 raft_consensus.cc:2454] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.173477 27524 leader_election.cc:302] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.173544 29241 raft_consensus.cc:2781] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.173566 29241 raft_consensus.cc:482] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.173583 29241 raft_consensus.cc:3037] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.173604 28656 tablet_bootstrap.cc:492] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.173640 28656 ts_tablet_manager.cc:1367] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.173758 28656 raft_consensus.cc:348] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.173794 28656 raft_consensus.cc:374] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.173803 28656 raft_consensus.cc:729] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.173835 28656 consensus_queue.cc:260] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.173926 28656 ts_tablet_manager.cc:1398] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.173974 28656 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.174125 29241 raft_consensus.cc:504] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.174229 29241 leader_election.cc:288] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.174268 28656 tablet_bootstrap.cc:654] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.174327 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26dbf9f67602403e8d0b1b2fb82acebc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.174379 27450 raft_consensus.cc:3037] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.174635 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26dbf9f67602403e8d0b1b2fb82acebc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.174686 27719 raft_consensus.cc:3037] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.174937 27450 raft_consensus.cc:2454] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.175019 28656 tablet_bootstrap.cc:492] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.175055 28656 ts_tablet_manager.cc:1367] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.175169 27719 raft_consensus.cc:2454] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.175174 28656 raft_consensus.cc:348] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.175225 28656 raft_consensus.cc:374] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.175244 28656 raft_consensus.cc:729] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.175271 28656 consensus_queue.cc:260] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.175364 28656 ts_tablet_manager.cc:1398] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.175453 27524 leader_election.cc:302] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.175542 29241 raft_consensus.cc:2781] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.175583 29241 raft_consensus.cc:686] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.175616 29241 consensus_queue.cc:237] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.176393 27196 catalog_manager.cc:5484] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.176609 28685 raft_consensus.cc:482] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.176659 28685 raft_consensus.cc:504] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.176748 28685 leader_election.cc:288] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.176895 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef574442d0ca4988b77635dd7e8ce5db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.176956 27719 raft_consensus.cc:2454] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.177131 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef574442d0ca4988b77635dd7e8ce5db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.177191 27582 raft_consensus.cc:2454] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.177284 27386 leader_election.cc:302] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.177393 28685 raft_consensus.cc:2781] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.177421 28685 raft_consensus.cc:482] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.177438 28685 raft_consensus.cc:3037] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.177975 28685 raft_consensus.cc:504] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.178068 28685 leader_election.cc:288] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.178231 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef574442d0ca4988b77635dd7e8ce5db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.178289 27582 raft_consensus.cc:3037] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.178736 27582 raft_consensus.cc:2454] T ef574442d0ca4988b77635dd7e8ce5db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.178820 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef574442d0ca4988b77635dd7e8ce5db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.178871 27719 raft_consensus.cc:3037] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.178864 27386 leader_election.cc:302] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.178925 28685 raft_consensus.cc:2781] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.178966 28685 raft_consensus.cc:686] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.179003 28685 consensus_queue.cc:237] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.179322 27719 raft_consensus.cc:2454] T ef574442d0ca4988b77635dd7e8ce5db P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.179809 27196 catalog_manager.cc:5484] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.184026 28685 raft_consensus.cc:482] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.184072 28685 raft_consensus.cc:504] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.184167 28685 leader_election.cc:288] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.184289 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb685fcf4a045f99b2cdebe5bf164d7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.184346 27582 raft_consensus.cc:2454] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.184410 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb685fcf4a045f99b2cdebe5bf164d7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.184451 27386 leader_election.cc:302] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.184480 27719 raft_consensus.cc:2454] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.184528 28685 raft_consensus.cc:2781] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.184567 28685 raft_consensus.cc:482] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.184585 28685 raft_consensus.cc:3037] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.184963 28685 raft_consensus.cc:504] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.185068 28685 leader_election.cc:288] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.185168 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb685fcf4a045f99b2cdebe5bf164d7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.185201 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fb685fcf4a045f99b2cdebe5bf164d7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.185227 27719 raft_consensus.cc:3037] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.185256 27582 raft_consensus.cc:3037] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.185420 28774 raft_consensus.cc:482] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.185467 28774 raft_consensus.cc:504] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.185564 28774 leader_election.cc:288] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.185752 27719 raft_consensus.cc:2454] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.185791 27582 raft_consensus.cc:2454] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.185853 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a1100e58af4ce980576a3e78b98600" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.185907 27585 raft_consensus.cc:2454] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.185976 27386 leader_election.cc:302] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.186020 27656 leader_election.cc:302] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.186062 28685 raft_consensus.cc:2781] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.186092 28774 raft_consensus.cc:2781] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.186105 28685 raft_consensus.cc:686] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.186125 28774 raft_consensus.cc:482] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.186141 28685 consensus_queue.cc:237] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.186163 28774 raft_consensus.cc:3037] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.186712 28774 raft_consensus.cc:504] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.186842 28774 leader_election.cc:288] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.186908 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a1100e58af4ce980576a3e78b98600" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.186955 27450 raft_consensus.cc:2454] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.187080 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a1100e58af4ce980576a3e78b98600" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.187124 27450 raft_consensus.cc:3037] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.187106 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a1100e58af4ce980576a3e78b98600" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.187150 27585 raft_consensus.cc:3037] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.187686 27585 raft_consensus.cc:2454] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.187686 27450 raft_consensus.cc:2454] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.187836 27659 leader_election.cc:302] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.187929 28774 raft_consensus.cc:2781] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.187974 28774 raft_consensus.cc:686] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.188010 28774 consensus_queue.cc:237] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.188043 27196 catalog_manager.cc:5484] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.189013 27201 catalog_manager.cc:5484] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.189102 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.190201 28685 raft_consensus.cc:482] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.190263 28685 raft_consensus.cc:504] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.190366 28685 leader_election.cc:288] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.190474 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fb59f7012094fab9cb807407fbc7996" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.190547 27719 raft_consensus.cc:2454] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.190610 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fb59f7012094fab9cb807407fbc7996" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.190659 27585 raft_consensus.cc:2454] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.190654 27388 leader_election.cc:302] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.190714 28685 raft_consensus.cc:2781] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.190737 28685 raft_consensus.cc:482] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.190753 28685 raft_consensus.cc:3037] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.191164 28685 raft_consensus.cc:504] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.191249 28685 leader_election.cc:288] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.191404 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fb59f7012094fab9cb807407fbc7996" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.191401 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fb59f7012094fab9cb807407fbc7996" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.191452 27585 raft_consensus.cc:3037] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.191453 27719 raft_consensus.cc:3037] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.191908 27719 raft_consensus.cc:2454] T 9fb59f7012094fab9cb807407fbc7996 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.191915 27585 raft_consensus.cc:2454] T 9fb59f7012094fab9cb807407fbc7996 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.192047 27388 leader_election.cc:302] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.192131 28685 raft_consensus.cc:2781] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.192166 28685 raft_consensus.cc:686] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.192198 28685 consensus_queue.cc:237] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.193042 27201 catalog_manager.cc:5484] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.198290 28766 raft_consensus.cc:482] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.198338 28766 raft_consensus.cc:504] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.198458 28766 leader_election.cc:288] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.198586 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa960c0281414895a12972c4c1e5269c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.198580 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa960c0281414895a12972c4c1e5269c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.198643 27719 raft_consensus.cc:2454] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.198644 27585 raft_consensus.cc:2454] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.198743 27386 leader_election.cc:302] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.198827 28766 raft_consensus.cc:2781] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.198868 28766 raft_consensus.cc:482] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.198894 28766 raft_consensus.cc:3037] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.199317 28766 raft_consensus.cc:504] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.199419 28766 leader_election.cc:288] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.199533 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa960c0281414895a12972c4c1e5269c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.199599 27719 raft_consensus.cc:3037] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.199990 27719 raft_consensus.cc:2454] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.200125 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa960c0281414895a12972c4c1e5269c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.200179 27585 raft_consensus.cc:3037] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.200567 27585 raft_consensus.cc:2454] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.200661 27388 leader_election.cc:302] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.200738 28766 raft_consensus.cc:2781] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.200796 28766 raft_consensus.cc:686] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.200840 28766 consensus_queue.cc:237] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.201506 27201 catalog_manager.cc:5484] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.203218 28766 consensus_queue.cc:1035] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.204232 28767 consensus_queue.cc:1035] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.205900 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7b6bac8ae6b04c248a79775c7570af75" } modify_external_catalogs: true
I20240503 02:09:16.205962 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7b6bac8ae6b04c248a79775c7570af75" } modify_external_catalogs: true
I20240503 02:09:16.206254 27585 raft_consensus.cc:1261] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.206481 27450 raft_consensus.cc:1261] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.206557 28693 consensus_queue.cc:1035] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.206723 28693 consensus_queue.cc:1035] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.206791 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 791439779c5b4f5f922403340c4b7a6f
I20240503 02:09:16.206980 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 791439779c5b4f5f922403340c4b7a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.207095 28654 tablet_replica.cc:330] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.207156 28654 raft_consensus.cc:2229] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.207186 28654 raft_consensus.cc:2258] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.207252 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 791439779c5b4f5f922403340c4b7a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.207369 28661 tablet_replica.cc:330] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.207372 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 791439779c5b4f5f922403340c4b7a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.207422 28661 raft_consensus.cc:2229] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.207453 28661 raft_consensus.cc:2258] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.207475 28653 tablet_replica.cc:330] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.207545 28653 raft_consensus.cc:2229] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.207589 28653 pending_rounds.cc:62] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:16.207605 28653 pending_rounds.cc:68] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420031636979712 op_type: NO_OP noop_request { }
I20240503 02:09:16.207636 28653 raft_consensus.cc:2866] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.207662 28653 raft_consensus.cc:2258] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.207648 28654 ts_tablet_manager.cc:1875] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.207746 28661 ts_tablet_manager.cc:1875] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.208770 28653 ts_tablet_manager.cc:1875] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.208981 28661 ts_tablet_manager.cc:1888] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.209017 28661 log.cc:1196] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/791439779c5b4f5f922403340c4b7a6f
I20240503 02:09:16.209160 28661 ts_tablet_manager.cc:1909] T 791439779c5b4f5f922403340c4b7a6f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.209262 28654 ts_tablet_manager.cc:1888] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.209306 28654 log.cc:1196] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/791439779c5b4f5f922403340c4b7a6f
I20240503 02:09:16.209478 28654 ts_tablet_manager.cc:1909] T 791439779c5b4f5f922403340c4b7a6f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.209893 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 791439779c5b4f5f922403340c4b7a6f (table default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]) successfully deleted
I20240503 02:09:16.209894 28767 consensus_queue.cc:1035] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.210040 28653 ts_tablet_manager.cc:1888] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.210080 28653 log.cc:1196] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/791439779c5b4f5f922403340c4b7a6f
I20240503 02:09:16.210253 28653 ts_tablet_manager.cc:1909] T 791439779c5b4f5f922403340c4b7a6f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.210449 28694 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.210548 28710 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.210870 29167 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.211525 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.211800 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 791439779c5b4f5f922403340c4b7a6f (table default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]) successfully deleted
I20240503 02:09:16.212033 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.212497 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 791439779c5b4f5f922403340c4b7a6f (table default.Table_7b6bac8ae6b04c248a79775c7570af75 [id=bb06cc6e251b41d4b45868df4e464eed]) successfully deleted
I20240503 02:09:16.212695 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f4a1100e58af4ce980576a3e78b98600 (table default.Table_87eb7af059c842d6b7b4c8e2be015384 [id=5ff15871699b4869b63a0187224a6cd7]). Expected version 1 got 0
I20240503 02:09:16.214802 28409 consensus_queue.cc:1035] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.221467 28774 consensus_queue.cc:1035] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.222690 28774 consensus_queue.cc:1035] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.223027 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0294e95bfb874319bf21ee6eb1b62d63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.223140 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0294e95bfb874319bf21ee6eb1b62d63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.225713 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 8f852b718f524467947b42f653f3530b (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.225755 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 8f852b718f524467947b42f653f3530b (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.225831 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f852b718f524467947b42f653f3530b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.225849 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f852b718f524467947b42f653f3530b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.225919 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 8f852b718f524467947b42f653f3530b (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.226002 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f852b718f524467947b42f653f3530b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.226887 28664 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.226967 28655 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.227241 28664 tablet_bootstrap.cc:654] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.227270 28655 tablet_bootstrap.cc:654] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.227242 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_665b0d3aca514edd91cee5dc4f99a3e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.227339 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_665b0d3aca514edd91cee5dc4f99a3e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.227675 28656 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.227855 28664 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.227887 28664 ts_tablet_manager.cc:1367] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.227952 28656 tablet_bootstrap.cc:654] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.227983 28664 raft_consensus.cc:348] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.228017 28664 raft_consensus.cc:374] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.228050 28664 raft_consensus.cc:729] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.228094 28664 consensus_queue.cc:260] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.228199 28664 ts_tablet_manager.cc:1398] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.228943 28656 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.228986 28656 ts_tablet_manager.cc:1367] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.229202 28656 raft_consensus.cc:348] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.229259 28656 raft_consensus.cc:374] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.229274 28656 raft_consensus.cc:729] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.229312 28656 consensus_queue.cc:260] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.229415 28656 ts_tablet_manager.cc:1398] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.229919 28774 raft_consensus.cc:482] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.229966 28774 raft_consensus.cc:504] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.230075 28774 leader_election.cc:288] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.230230 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5b3f2a01b5453983b0d767ff376a6c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.230295 27450 raft_consensus.cc:2454] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.230476 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5b3f2a01b5453983b0d767ff376a6c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.230548 27585 raft_consensus.cc:2454] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.230577 27659 leader_election.cc:302] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.230731 28774 raft_consensus.cc:2781] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.230768 28774 raft_consensus.cc:482] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.230787 28774 raft_consensus.cc:3037] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.230813 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d52224df3392476fa9a4959341cc737a (DEFAULT_TABLE table=default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.230844 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d52224df3392476fa9a4959341cc737a (DEFAULT_TABLE table=default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.230895 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52224df3392476fa9a4959341cc737a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.230919 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52224df3392476fa9a4959341cc737a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.231381 28774 raft_consensus.cc:504] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.231482 28774 leader_election.cc:288] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.231721 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5b3f2a01b5453983b0d767ff376a6c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.231801 27450 raft_consensus.cc:3037] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.231853 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_57f253fedf2c4938996c82cadded8531" } modify_external_catalogs: true
I20240503 02:09:16.231922 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_57f253fedf2c4938996c82cadded8531" } modify_external_catalogs: true
I20240503 02:09:16.232409 27450 raft_consensus.cc:2454] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.232533 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5b3f2a01b5453983b0d767ff376a6c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.232563 27659 leader_election.cc:302] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.232592 27585 raft_consensus.cc:3037] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.232630 28774 raft_consensus.cc:2781] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.232661 28774 raft_consensus.cc:686] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.232692 28774 consensus_queue.cc:237] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.232883 28656 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.233124 27585 raft_consensus.cc:2454] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.233207 28656 tablet_bootstrap.cc:654] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.233481 27703 tablet_service.cc:1467] Processing CreateTablet for tablet d52224df3392476fa9a4959341cc737a (DEFAULT_TABLE table=default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.233577 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d52224df3392476fa9a4959341cc737a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.233762 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1c08a7da01c54148bc19f3b29c00f841
I20240503 02:09:16.234022 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.234100 28654 tablet_replica.cc:330] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.234158 28654 raft_consensus.cc:2229] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.234213 28654 raft_consensus.cc:2258] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.234371 28654 ts_tablet_manager.cc:1875] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.234539 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.234625 28664 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.234673 28656 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.234707 28656 ts_tablet_manager.cc:1367] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.234818 28656 raft_consensus.cc:348] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.234867 28656 raft_consensus.cc:374] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.234884 28656 raft_consensus.cc:729] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.234884 28664 tablet_bootstrap.cc:654] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.234921 28656 consensus_queue.cc:260] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.235018 28656 ts_tablet_manager.cc:1398] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.235163 28653 tablet_replica.cc:330] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.235224 28653 raft_consensus.cc:2229] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.235252 28653 raft_consensus.cc:2258] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.235582 28654 ts_tablet_manager.cc:1888] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.235621 28654 log.cc:1196] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1c08a7da01c54148bc19f3b29c00f841
I20240503 02:09:16.235790 28654 ts_tablet_manager.cc:1909] T 1c08a7da01c54148bc19f3b29c00f841 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.236073 28653 ts_tablet_manager.cc:1875] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.236053 28410 consensus_queue.cc:1035] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.236071 27202 catalog_manager.cc:5484] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.236151 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:16.236899 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1c08a7da01c54148bc19f3b29c00f841 (table default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]) successfully deleted
I20240503 02:09:16.237185 28653 ts_tablet_manager.cc:1888] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.237228 28653 log.cc:1196] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1c08a7da01c54148bc19f3b29c00f841
I20240503 02:09:16.237371 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_97b99c616e314712a650a66d5dd873cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.237432 28653 ts_tablet_manager.cc:1909] T 1c08a7da01c54148bc19f3b29c00f841 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
W20240503 02:09:16.237462 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97b99c616e314712a650a66d5dd873cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.237819 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1c08a7da01c54148bc19f3b29c00f841 (table default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]) successfully deleted
I20240503 02:09:16.237864 28655 tablet_bootstrap.cc:492] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.237903 28655 ts_tablet_manager.cc:1367] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:16.238011 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_28e4b24a6e884bb6a60d26a9ae69d82a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.238042 28655 raft_consensus.cc:348] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.238088 28655 raft_consensus.cc:374] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20240503 02:09:16.238093 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28e4b24a6e884bb6a60d26a9ae69d82a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.238106 28655 raft_consensus.cc:729] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.238142 28655 consensus_queue.cc:260] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.238219 28655 ts_tablet_manager.cc:1398] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.238269 28655 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.238582 28655 tablet_bootstrap.cc:654] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.238698 29241 raft_consensus.cc:482] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.238742 29241 raft_consensus.cc:504] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.238765 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c08a7da01c54148bc19f3b29c00f841 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.238843 28661 tablet_replica.cc:330] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.238850 29241 leader_election.cc:288] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.238898 28661 raft_consensus.cc:2229] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.238927 28661 raft_consensus.cc:2258] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.238998 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce0b8f7896e400c8847800bccdda58f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.239042 28664 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.239058 27719 raft_consensus.cc:2454] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.239074 28664 ts_tablet_manager.cc:1367] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:16.239116 28661 ts_tablet_manager.cc:1875] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.239187 27523 leader_election.cc:302] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.239203 28664 raft_consensus.cc:348] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.239250 28664 raft_consensus.cc:374] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.239253 29241 raft_consensus.cc:2781] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.239269 28664 raft_consensus.cc:729] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.239279 29241 raft_consensus.cc:482] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.239296 29241 raft_consensus.cc:3037] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.239296 28664 consensus_queue.cc:260] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.239373 28664 ts_tablet_manager.cc:1398] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.239864 29241 raft_consensus.cc:504] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.239969 29241 leader_election.cc:288] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.240181 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce0b8f7896e400c8847800bccdda58f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.240234 27719 raft_consensus.cc:3037] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.240309 28661 ts_tablet_manager.cc:1888] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.240351 28661 log.cc:1196] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1c08a7da01c54148bc19f3b29c00f841
I20240503 02:09:16.240406 28655 tablet_bootstrap.cc:492] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.240438 28655 ts_tablet_manager.cc:1367] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.240532 28661 ts_tablet_manager.cc:1909] T 1c08a7da01c54148bc19f3b29c00f841 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.240551 28655 raft_consensus.cc:348] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.240599 28655 raft_consensus.cc:374] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.240617 28655 raft_consensus.cc:729] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.240649 28655 consensus_queue.cc:260] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.240739 28655 ts_tablet_manager.cc:1398] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.240770 27719 raft_consensus.cc:2454] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.240931 27523 leader_election.cc:302] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.241725 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d89add66b30d49348552022b8f5f8ce3 (DEFAULT_TABLE table=default.Table_97b99c616e314712a650a66d5dd873cb [id=8a42618c7bc44481a535fe91c3ede7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.241807 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89add66b30d49348552022b8f5f8ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.241832 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d89add66b30d49348552022b8f5f8ce3 (DEFAULT_TABLE table=default.Table_97b99c616e314712a650a66d5dd873cb [id=8a42618c7bc44481a535fe91c3ede7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.241923 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89add66b30d49348552022b8f5f8ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.241989 29241 raft_consensus.cc:2781] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.242030 29241 raft_consensus.cc:686] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.242063 29241 consensus_queue.cc:237] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.242884 28655 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.242941 28656 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.243249 28655 tablet_bootstrap.cc:654] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.243265 28656 tablet_bootstrap.cc:654] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.243588 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce0b8f7896e400c8847800bccdda58f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.243654 27450 raft_consensus.cc:2454] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.243589 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce0b8f7896e400c8847800bccdda58f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.243748 27453 raft_consensus.cc:3037] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.244002 29240 consensus_queue.cc:1035] T c76ce5e466e64b5cae709aecaa1f4c0e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.244022 27201 catalog_manager.cc:5484] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.244330 27453 raft_consensus.cc:2454] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.244428 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1c08a7da01c54148bc19f3b29c00f841 (table default.Table_57f253fedf2c4938996c82cadded8531 [id=e93808abff8c4f889ca2987b7a5f6946]) successfully deleted
I20240503 02:09:16.244685 27703 tablet_service.cc:1467] Processing CreateTablet for tablet d89add66b30d49348552022b8f5f8ce3 (DEFAULT_TABLE table=default.Table_97b99c616e314712a650a66d5dd873cb [id=8a42618c7bc44481a535fe91c3ede7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.244781 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89add66b30d49348552022b8f5f8ce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.245544 28656 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.245581 28656 ts_tablet_manager.cc:1367] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.245731 28656 raft_consensus.cc:348] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.245781 28656 raft_consensus.cc:374] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.245797 28656 raft_consensus.cc:729] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.245824 28656 consensus_queue.cc:260] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.245901 28656 ts_tablet_manager.cc:1398] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.246105 28664 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.246490 28664 tablet_bootstrap.cc:654] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.247534 28655 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.247578 28655 ts_tablet_manager.cc:1367] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.247694 27560 tablet_service.cc:1467] Processing CreateTablet for tablet b3eb52c8c0a94673bf1546f71320a6fe (DEFAULT_TABLE table=default.Table_28e4b24a6e884bb6a60d26a9ae69d82a [id=7515b8bcce32455fac410e29a6b35adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.247699 28655 raft_consensus.cc:348] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.247753 28655 raft_consensus.cc:374] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.247772 28655 raft_consensus.cc:729] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.247779 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3eb52c8c0a94673bf1546f71320a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.247807 28655 consensus_queue.cc:260] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.247890 28655 ts_tablet_manager.cc:1398] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.248329 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b3eb52c8c0a94673bf1546f71320a6fe (DEFAULT_TABLE table=default.Table_28e4b24a6e884bb6a60d26a9ae69d82a [id=7515b8bcce32455fac410e29a6b35adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.248442 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3eb52c8c0a94673bf1546f71320a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.248746 28656 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.249073 28656 tablet_bootstrap.cc:654] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.249524 28656 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.249562 28656 ts_tablet_manager.cc:1367] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.249609 27427 tablet_service.cc:1467] Processing CreateTablet for tablet b3eb52c8c0a94673bf1546f71320a6fe (DEFAULT_TABLE table=default.Table_28e4b24a6e884bb6a60d26a9ae69d82a [id=7515b8bcce32455fac410e29a6b35adf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.249684 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3eb52c8c0a94673bf1546f71320a6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.249689 28656 raft_consensus.cc:348] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.249737 28656 raft_consensus.cc:374] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.249755 28656 raft_consensus.cc:729] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.249790 28656 consensus_queue.cc:260] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.249914 28656 ts_tablet_manager.cc:1398] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.250161 28664 tablet_bootstrap.cc:492] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.250200 28664 ts_tablet_manager.cc:1367] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.250326 28664 raft_consensus.cc:348] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.250373 28664 raft_consensus.cc:374] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.250387 28664 raft_consensus.cc:729] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.250425 28664 consensus_queue.cc:260] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.250502 28664 ts_tablet_manager.cc:1398] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.250573 28664 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.250919 28664 tablet_bootstrap.cc:654] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.251108 28655 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.251466 28655 tablet_bootstrap.cc:654] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.251688 27853 raft_consensus.cc:482] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.251730 27853 raft_consensus.cc:504] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.251816 27853 leader_election.cc:288] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.251967 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe14cc0844c2b842ba27e21451535" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.252019 27585 raft_consensus.cc:2454] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.252153 27656 leader_election.cc:302] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.252319 28655 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.252367 28655 ts_tablet_manager.cc:1367] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.252501 28655 raft_consensus.cc:348] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.252549 28655 raft_consensus.cc:374] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.252565 28655 raft_consensus.cc:729] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.252596 28655 consensus_queue.cc:260] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.252671 28655 ts_tablet_manager.cc:1398] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.252806 28774 raft_consensus.cc:2781] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.252838 28774 raft_consensus.cc:482] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.252854 28774 raft_consensus.cc:3037] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:16.252877 27201 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:16.253371 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe14cc0844c2b842ba27e21451535" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.253427 27453 raft_consensus.cc:2454] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.253808 28774 raft_consensus.cc:504] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.253907 28774 leader_election.cc:288] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.253916 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.254007 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.254091 28664 tablet_bootstrap.cc:492] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.254125 28664 ts_tablet_manager.cc:1367] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.254142 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe14cc0844c2b842ba27e21451535" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.254191 27453 raft_consensus.cc:3037] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.254304 28664 raft_consensus.cc:348] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.254348 28664 raft_consensus.cc:374] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.254364 28664 raft_consensus.cc:729] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.254403 28664 consensus_queue.cc:260] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.254478 28664 ts_tablet_manager.cc:1398] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.254771 27453 raft_consensus.cc:2454] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.254896 27659 leader_election.cc:302] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.254966 28774 raft_consensus.cc:2781] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.254999 28774 raft_consensus.cc:686] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.254978 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "322fe14cc0844c2b842ba27e21451535" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.255046 27585 raft_consensus.cc:3037] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.255033 28774 consensus_queue.cc:237] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.255604 27585 raft_consensus.cc:2454] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.256004 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_36a299ced16c477a914bad735bb8b7d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.256068 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7de06993e4e948b3bf97dad177b17e83" } new_table_name: "default.Table_ddee9f12244d4eb2b35bd506b46a46a3" modify_external_catalogs: true
W20240503 02:09:16.256096 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36a299ced16c477a914bad735bb8b7d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.256201 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_445a2ff5353846579f6884b81b92e97a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.256264 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_445a2ff5353846579f6884b81b92e97a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.256606 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 82e659f5818744f584e64997789ebf86 (DEFAULT_TABLE table=default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7 [id=c10aff1a98f64f13a7ffb57bf8a6e815]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.256688 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e659f5818744f584e64997789ebf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.256882 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 82e659f5818744f584e64997789ebf86 (DEFAULT_TABLE table=default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7 [id=c10aff1a98f64f13a7ffb57bf8a6e815]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.256956 27585 raft_consensus.cc:1261] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.256968 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e659f5818744f584e64997789ebf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.257485 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_fe126d10909141ffaf718841bfe77ce2" } modify_external_catalogs: true
I20240503 02:09:16.257547 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_fe126d10909141ffaf718841bfe77ce2" } modify_external_catalogs: true
I20240503 02:09:16.258040 27853 raft_consensus.cc:482] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.258081 27853 raft_consensus.cc:504] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.258162 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 82e659f5818744f584e64997789ebf86 (DEFAULT_TABLE table=default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7 [id=c10aff1a98f64f13a7ffb57bf8a6e815]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.258167 27198 catalog_manager.cc:5484] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.258173 27853 leader_election.cc:288] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.258250 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82e659f5818744f584e64997789ebf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.258267 27853 consensus_queue.cc:1035] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.258457 28656 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.258574 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b66e153765c42128c4f64f0894fbe87" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.258631 27450 raft_consensus.cc:2454] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.258728 27659 leader_election.cc:302] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.258780 28656 tablet_bootstrap.cc:654] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.258790 27853 raft_consensus.cc:2781] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.258816 27853 raft_consensus.cc:482] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.258831 27853 raft_consensus.cc:3037] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.258867 27453 raft_consensus.cc:1261] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.259016 28740 consensus_queue.cc:1035] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.259470 27853 raft_consensus.cc:504] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.259562 27853 leader_election.cc:288] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.259827 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b66e153765c42128c4f64f0894fbe87" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.259894 27587 raft_consensus.cc:3037] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.260324 28664 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.260494 27587 raft_consensus.cc:2454] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.260650 28664 tablet_bootstrap.cc:654] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.260813 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_68e788fa579b47838b1c68cd3cfde502"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.260900 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d45db7c30b55445d89b52cacbf36d158"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.260977 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d45db7c30b55445d89b52cacbf36d158 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:16.260910 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68e788fa579b47838b1c68cd3cfde502 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.261077 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b66e153765c42128c4f64f0894fbe87" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.261134 27450 raft_consensus.cc:3037] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.261236 28655 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.261379 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:16.261693 28655 tablet_bootstrap.cc:654] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.262012 27656 leader_election.cc:302] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.261700 27450 raft_consensus.cc:2454] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.262272 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b66e153765c42128c4f64f0894fbe87" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.262326 27585 raft_consensus.cc:2362] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:16.262392 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet cdab84ce3fd14cf3a7fdb838cfe9e648
I20240503 02:09:16.262655 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:16.262832 28656 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.262825 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.262867 28656 ts_tablet_manager.cc:1367] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:16.262917 28661 tablet_replica.cc:330] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.262984 28661 raft_consensus.cc:2229] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.263015 28661 raft_consensus.cc:2258] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.263020 28656 raft_consensus.cc:348] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.263073 28656 raft_consensus.cc:374] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.263090 28656 raft_consensus.cc:729] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.263098 27853 raft_consensus.cc:2781] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.263134 27853 raft_consensus.cc:686] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.263126 28656 consensus_queue.cc:260] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.263165 27853 consensus_queue.cc:237] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.263214 28656 ts_tablet_manager.cc:1398] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.263284 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.263362 28694 tablet.cc:1722] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.263584 28664 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.263620 28664 ts_tablet_manager.cc:1367] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.263689 28654 tablet_replica.cc:330] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.263746 28654 raft_consensus.cc:2229] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.263782 28654 raft_consensus.cc:2258] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.263751 28664 raft_consensus.cc:348] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.263820 28664 raft_consensus.cc:374] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.263839 28664 raft_consensus.cc:729] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.263870 28664 consensus_queue.cc:260] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.263958 28664 ts_tablet_manager.cc:1398] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.263762 28661 ts_tablet_manager.cc:1875] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.264847 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet cdab84ce3fd14cf3a7fdb838cfe9e648 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.264915 28653 tablet_replica.cc:330] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.264963 28653 raft_consensus.cc:2229] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.265007 28653 raft_consensus.cc:2258] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.265260 28653 ts_tablet_manager.cc:1875] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.265512 28710 tablet.cc:1722] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.265631 28661 ts_tablet_manager.cc:1888] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.265673 28661 log.cc:1196] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/cdab84ce3fd14cf3a7fdb838cfe9e648
I20240503 02:09:16.265841 28661 ts_tablet_manager.cc:1909] T cdab84ce3fd14cf3a7fdb838cfe9e648 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.265836 28654 ts_tablet_manager.cc:1875] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.266196 27719 raft_consensus.cc:1261] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.266335 27848 consensus_queue.cc:1035] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.267081 28654 ts_tablet_manager.cc:1888] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.267119 28654 log.cc:1196] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/cdab84ce3fd14cf3a7fdb838cfe9e648
I20240503 02:09:16.267287 28654 ts_tablet_manager.cc:1909] T cdab84ce3fd14cf3a7fdb838cfe9e648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.267995 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8d8edf43434e5597e58084babb7843 (DEFAULT_TABLE table=default.Table_445a2ff5353846579f6884b81b92e97a [id=da76a80d32d34ee0b39f826d114a51ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.268090 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8d8edf43434e5597e58084babb7843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.268110 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 409fa97790fe47f89b6d95a392e2e334 (DEFAULT_TABLE table=default.Table_36a299ced16c477a914bad735bb8b7d6 [id=f7d862ad0364458ba75b38483172b371]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.268208 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 409fa97790fe47f89b6d95a392e2e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.269133 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 409fa97790fe47f89b6d95a392e2e334 (DEFAULT_TABLE table=default.Table_36a299ced16c477a914bad735bb8b7d6 [id=f7d862ad0364458ba75b38483172b371]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.269217 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 409fa97790fe47f89b6d95a392e2e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.269263 29167 tablet.cc:1722] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.269371 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8d8edf43434e5597e58084babb7843 (DEFAULT_TABLE table=default.Table_445a2ff5353846579f6884b81b92e97a [id=da76a80d32d34ee0b39f826d114a51ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.269447 27453 raft_consensus.cc:1261] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.269450 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8d8edf43434e5597e58084babb7843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.269618 28664 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.269961 28664 tablet_bootstrap.cc:654] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.270026 28693 raft_consensus.cc:482] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.270066 28693 raft_consensus.cc:504] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.270159 28693 leader_election.cc:288] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.270354 29240 consensus_queue.cc:1035] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.270726 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52224df3392476fa9a4959341cc737a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.270783 27449 raft_consensus.cc:2454] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.270992 27659 leader_election.cc:302] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.271095 28774 raft_consensus.cc:2781] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.271131 28774 raft_consensus.cc:482] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.271149 28774 raft_consensus.cc:3037] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.271687 28653 ts_tablet_manager.cc:1888] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.271729 28653 log.cc:1196] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/cdab84ce3fd14cf3a7fdb838cfe9e648
I20240503 02:09:16.271720 28774 raft_consensus.cc:504] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.271816 28774 leader_election.cc:288] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.271903 28653 ts_tablet_manager.cc:1909] T cdab84ce3fd14cf3a7fdb838cfe9e648 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.271998 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52224df3392476fa9a4959341cc737a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.272053 27585 raft_consensus.cc:2454] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.272096 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52224df3392476fa9a4959341cc737a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.272126 27585 raft_consensus.cc:3037] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.272445 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 23129af1b17c45e093e85c2b0032f864 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.272675 27585 raft_consensus.cc:2454] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.272691 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 409fa97790fe47f89b6d95a392e2e334 (DEFAULT_TABLE table=default.Table_36a299ced16c477a914bad735bb8b7d6 [id=f7d862ad0364458ba75b38483172b371]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.272675 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_68e788fa579b47838b1c68cd3cfde502" } new_table_name: "default.Table_1b09b97f30ac43ecbbeac65f8366738a" modify_external_catalogs: true
I20240503 02:09:16.272771 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 409fa97790fe47f89b6d95a392e2e334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.272907 27656 leader_election.cc:302] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.273068 28693 raft_consensus.cc:2781] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.273109 28693 raft_consensus.cc:686] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.273144 28693 consensus_queue.cc:237] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.273452 28698 tablet.cc:1722] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.273916 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8d8edf43434e5597e58084babb7843 (DEFAULT_TABLE table=default.Table_445a2ff5353846579f6884b81b92e97a [id=da76a80d32d34ee0b39f826d114a51ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.273995 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8d8edf43434e5597e58084babb7843. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.274011 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 23129af1b17c45e093e85c2b0032f864
I20240503 02:09:16.274089 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 23129af1b17c45e093e85c2b0032f864 deleted and replaced by tablet 24e32edb5ccf44199fce2ec6a2c8cf63
I20240503 02:09:16.274123 28656 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:16.274441 27197 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b16f1976661e4dba8b366773533ca59e consensus state has no leader
I20240503 02:09:16.274461 28656 tablet_bootstrap.cc:654] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.274680 28664 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.274714 28664 ts_tablet_manager.cc:1367] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.274737 27202 catalog_manager.cc:5484] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.274822 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:16.274833 28664 raft_consensus.cc:348] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.274879 28664 raft_consensus.cc:374] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.274896 28664 raft_consensus.cc:729] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.274931 28664 consensus_queue.cc:260] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.275024 28664 ts_tablet_manager.cc:1398] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.275070 28664 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.275382 28664 tablet_bootstrap.cc:654] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.275472 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 23129af1b17c45e093e85c2b0032f864 with delete_type TABLET_DATA_DELETED (replaced by tablet 24e32edb5ccf44199fce2ec6a2c8cf63) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.275506 28694 tablet.cc:1722] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.275643 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b16f1976661e4dba8b366773533ca59e (DEFAULT_TABLE table=default.Table_68e788fa579b47838b1c68cd3cfde502 [id=4b102e807c7a4aab83ecf1d206d8f80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.275729 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f1976661e4dba8b366773533ca59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.276283 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d52224df3392476fa9a4959341cc737a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.276336 27450 raft_consensus.cc:3037] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.276875 27450 raft_consensus.cc:2454] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.277079 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 23129af1b17c45e093e85c2b0032f864 with delete_type TABLET_DATA_DELETED (replaced by tablet 24e32edb5ccf44199fce2ec6a2c8cf63) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.277081 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 5b97787b7d314c39b4ea9e599dbe692e (DEFAULT_TABLE table=default.Table_d45db7c30b55445d89b52cacbf36d158 [id=8841b738f075497e9761d1c46151d168]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.277117 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2d4d8ce484e048a1b6b35bdbfac1a691"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.277210 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b97787b7d314c39b4ea9e599dbe692e. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:16.277212 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.277465 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_36a299ced16c477a914bad735bb8b7d6" } new_table_name: "default.Table_cbcba1cbea9d4431aadd54adea024b1e" modify_external_catalogs: true
I20240503 02:09:16.277483 28655 tablet_bootstrap.cc:492] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.277527 28655 ts_tablet_manager.cc:1367] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:16.277622 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b16f1976661e4dba8b366773533ca59e (DEFAULT_TABLE table=default.Table_68e788fa579b47838b1c68cd3cfde502 [id=4b102e807c7a4aab83ecf1d206d8f80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.277650 28655 raft_consensus.cc:348] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.277698 28655 raft_consensus.cc:374] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.277704 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f1976661e4dba8b366773533ca59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.277714 28655 raft_consensus.cc:729] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.277752 28655 consensus_queue.cc:260] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.277830 28655 ts_tablet_manager.cc:1398] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.277889 28655 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.278070 27562 tablet_service.cc:1467] Processing CreateTablet for tablet b16f1976661e4dba8b366773533ca59e (DEFAULT_TABLE table=default.Table_68e788fa579b47838b1c68cd3cfde502 [id=4b102e807c7a4aab83ecf1d206d8f80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.278154 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b16f1976661e4dba8b366773533ca59e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.278213 28655 tablet_bootstrap.cc:654] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.278517 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 5b97787b7d314c39b4ea9e599dbe692e (DEFAULT_TABLE table=default.Table_d45db7c30b55445d89b52cacbf36d158 [id=8841b738f075497e9761d1c46151d168]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.278601 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b97787b7d314c39b4ea9e599dbe692e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.278928 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (table default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7]) successfully deleted
I20240503 02:09:16.279107 28661 tablet_replica.cc:330] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.279158 28661 raft_consensus.cc:2229] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.279189 28661 raft_consensus.cc:2258] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.279402 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (table default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7]) successfully deleted
I20240503 02:09:16.279817 28656 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.279855 28656 ts_tablet_manager.cc:1367] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:16.279871 28664 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.279863 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_91792da8cc4444c1b2910bf6dda7b4fb" } new_table_name: "default.Table_057771b8908c4df3be472ab48352e93a" modify_external_catalogs: true
I20240503 02:09:16.279908 28664 ts_tablet_manager.cc:1367] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.279990 28656 raft_consensus.cc:348] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.280046 28656 raft_consensus.cc:374] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.280030 28664 raft_consensus.cc:348] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.280066 28656 raft_consensus.cc:729] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.280081 28664 raft_consensus.cc:374] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.280099 28664 raft_consensus.cc:729] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.280100 28656 consensus_queue.cc:260] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.280135 28664 consensus_queue.cc:260] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.280176 28656 ts_tablet_manager.cc:1398] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.280184 28653 tablet_replica.cc:330] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.280223 28664 ts_tablet_manager.cc:1398] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.280232 28656 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.280234 28653 raft_consensus.cc:2229] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.280269 28664 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.280279 28653 raft_consensus.cc:2258] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.280532 28656 tablet_bootstrap.cc:654] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.280681 28664 tablet_bootstrap.cc:654] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.280795 28655 tablet_bootstrap.cc:492] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.280830 28655 ts_tablet_manager.cc:1367] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.280947 28655 raft_consensus.cc:348] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.281000 28655 raft_consensus.cc:374] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.281018 28655 raft_consensus.cc:729] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.281049 28655 consensus_queue.cc:260] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.281126 28655 ts_tablet_manager.cc:1398] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.281172 28655 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.281343 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet cdab84ce3fd14cf3a7fdb838cfe9e648 (table default.Table_fe126d10909141ffaf718841bfe77ce2 [id=0805af74bdea4546b84d729e82bcfec7]) successfully deleted
I20240503 02:09:16.281535 28655 tablet_bootstrap.cc:654] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.281592 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 23129af1b17c45e093e85c2b0032f864 with delete_type TABLET_DATA_DELETED (replaced by tablet 24e32edb5ccf44199fce2ec6a2c8cf63) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.281659 28654 tablet_replica.cc:330] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.281701 28654 raft_consensus.cc:2229] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.281733 28654 raft_consensus.cc:2258] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.281973 28654 ts_tablet_manager.cc:1875] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.282012 28655 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.282047 28655 ts_tablet_manager.cc:1367] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.282166 28655 raft_consensus.cc:348] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.282214 28655 raft_consensus.cc:374] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.282231 28655 raft_consensus.cc:729] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.282261 28655 consensus_queue.cc:260] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.282341 28655 ts_tablet_manager.cc:1398] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.282385 28655 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.282577 28664 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.282629 28664 ts_tablet_manager.cc:1367] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.282752 28664 raft_consensus.cc:348] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.282800 28664 raft_consensus.cc:374] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.282817 28664 raft_consensus.cc:729] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.282848 28655 tablet_bootstrap.cc:654] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.282851 28664 consensus_queue.cc:260] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.282949 28664 ts_tablet_manager.cc:1398] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.283012 28664 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.283012 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 5b97787b7d314c39b4ea9e599dbe692e (DEFAULT_TABLE table=default.Table_d45db7c30b55445d89b52cacbf36d158 [id=8841b738f075497e9761d1c46151d168]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.283110 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b97787b7d314c39b4ea9e599dbe692e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.283313 28664 tablet_bootstrap.cc:654] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.283357 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6ce0b8f7896e400c8847800bccdda58f (table default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d]). Expected version 1 got 0
I20240503 02:09:16.285727 28664 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.285761 28664 ts_tablet_manager.cc:1367] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.285879 28664 raft_consensus.cc:348] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.285928 28664 raft_consensus.cc:374] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.285943 28664 raft_consensus.cc:729] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.285975 28664 consensus_queue.cc:260] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.286054 28664 ts_tablet_manager.cc:1398] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.286756 28694 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.286880 28655 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.286912 28655 ts_tablet_manager.cc:1367] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:16.287036 28655 raft_consensus.cc:348] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.287076 28686 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.287082 28655 raft_consensus.cc:374] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.287174 28655 raft_consensus.cc:729] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.287209 28655 consensus_queue.cc:260] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.287317 28655 ts_tablet_manager.cc:1398] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.287407 28655 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.287770 28655 tablet_bootstrap.cc:654] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.287835 29167 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.288380 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet be7a66cc87834e739e958a2caf73de0c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.289148 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.289227 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a166dbdc3574a0183e3cc4896307be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.289430 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6ce0b8f7896e400c8847800bccdda58f (table default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d]). Expected version 1 got 0
I20240503 02:09:16.289521 27198 catalog_manager.cc:5484] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.289724 28661 ts_tablet_manager.cc:1875] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.290270 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.290270 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.290349 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32edb5ccf44199fce2ec6a2c8cf63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.290381 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a166dbdc3574a0183e3cc4896307be2. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:16.290416 27197 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 409fa97790fe47f89b6d95a392e2e334 consensus state has no leader
I20240503 02:09:16.290531 28653 ts_tablet_manager.cc:1875] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.290719 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.290796 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32edb5ccf44199fce2ec6a2c8cf63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.291401 27853 raft_consensus.cc:482] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.291442 27853 raft_consensus.cc:504] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.291519 27853 leader_election.cc:288] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.291750 28653 ts_tablet_manager.cc:1888] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.291796 28653 log.cc:1196] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/23129af1b17c45e093e85c2b0032f864
I20240503 02:09:16.291898 28664 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.291898 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.291954 27585 raft_consensus.cc:2454] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.292196 28653 ts_tablet_manager.cc:1909] T 23129af1b17c45e093e85c2b0032f864 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.292248 28664 tablet_bootstrap.cc:654] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.292662 27656 leader_election.cc:302] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.292721 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d] alter complete (version 1)
I20240503 02:09:16.292799 28409 raft_consensus.cc:482] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.292837 28409 raft_consensus.cc:504] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.292912 28664 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.292924 28409 leader_election.cc:288] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.292943 28664 ts_tablet_manager.cc:1367] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.292948 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:16.293059 28664 raft_consensus.cc:348] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.293109 28664 raft_consensus.cc:374] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.293073 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.293128 28664 raft_consensus.cc:729] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.293078 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.293139 27585 raft_consensus.cc:2454] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.293152 27449 raft_consensus.cc:2454] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.293164 28664 consensus_queue.cc:260] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.293241 28664 ts_tablet_manager.cc:1398] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.293287 28664 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.293388 27853 raft_consensus.cc:2781] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.293421 27853 raft_consensus.cc:482] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.293438 27853 raft_consensus.cc:3037] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.293455 28654 ts_tablet_manager.cc:1888] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.293498 28654 log.cc:1196] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/23129af1b17c45e093e85c2b0032f864
I20240503 02:09:16.293668 28654 ts_tablet_manager.cc:1909] T 23129af1b17c45e093e85c2b0032f864 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.293727 28664 tablet_bootstrap.cc:654] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.294040 27853 raft_consensus.cc:504] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.294148 27853 leader_election.cc:288] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.294153 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:16.294342 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.294380 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.294399 27719 raft_consensus.cc:2379] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:16.294452 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a166dbdc3574a0183e3cc4896307be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.294590 27386 leader_election.cc:302] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.294654 28409 raft_consensus.cc:2781] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.294675 28409 raft_consensus.cc:482] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.294670 28661 ts_tablet_manager.cc:1888] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.294693 28409 raft_consensus.cc:3037] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.294716 28661 log.cc:1196] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/23129af1b17c45e093e85c2b0032f864
I20240503 02:09:16.294888 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.294905 28661 ts_tablet_manager.cc:1909] T 23129af1b17c45e093e85c2b0032f864 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.294979 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e32edb5ccf44199fce2ec6a2c8cf63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.295270 28409 raft_consensus.cc:504] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.295363 28409 leader_election.cc:288] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.295755 28656 tablet_bootstrap.cc:492] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.295792 28656 ts_tablet_manager.cc:1367] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240503 02:09:16.295938 28656 raft_consensus.cc:348] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.295989 28656 raft_consensus.cc:374] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.296005 28656 raft_consensus.cc:729] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.296033 28656 consensus_queue.cc:260] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.296113 28656 ts_tablet_manager.cc:1398] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.296169 28656 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.296244 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 23129af1b17c45e093e85c2b0032f864 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:16.296471 28656 tablet_bootstrap.cc:654] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.296528 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_69373c98e90d49f787b21be369651a47" } new_table_name: "default.Table_603bc38c1af74b51a2983f8d604e5eb5" modify_external_catalogs: true
I20240503 02:09:16.296671 28740 raft_consensus.cc:482] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.296691 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.296715 28740 raft_consensus.cc:504] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.296747 27720 raft_consensus.cc:2379] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:16.296802 28740 leader_election.cc:288] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.296788 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.296837 27450 raft_consensus.cc:2379] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c2be543b8aad4e80a58d6b4ab669faae in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:16.297025 29241 raft_consensus.cc:482] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.297062 29241 raft_consensus.cc:504] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.297084 27848 consensus_queue.cc:1035] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.297148 29241 leader_election.cc:288] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.297168 27848 consensus_queue.cc:1035] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.297181 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_057771b8908c4df3be472ab48352e93a [id=61076672a8034d1ebb1077d0795360d1] alter complete (version 1)
I20240503 02:09:16.297235 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.297269 27585 raft_consensus.cc:3037] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.297348 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8424c39f615456e8db186a9c124629c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.297398 27449 raft_consensus.cc:2454] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.297518 28664 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.297521 27524 leader_election.cc:302] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.297554 28664 ts_tablet_manager.cc:1367] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:16.297598 28410 raft_consensus.cc:2781] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.297624 28410 raft_consensus.cc:482] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.297641 28410 raft_consensus.cc:3037] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.297670 28664 raft_consensus.cc:348] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.297717 28664 raft_consensus.cc:374] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.297734 28664 raft_consensus.cc:729] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.297768 28664 consensus_queue.cc:260] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.297814 27585 raft_consensus.cc:2454] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.297837 28664 ts_tablet_manager.cc:1398] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.297899 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60368acf76cd4e9eb8e6750b37ad95b9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.297942 27582 raft_consensus.cc:2379] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c2be543b8aad4e80a58d6b4ab669faae in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:16.297981 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f1976661e4dba8b366773533ca59e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.298245 28410 raft_consensus.cc:504] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.298310 28410 leader_election.cc:288] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.298291 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
I20240503 02:09:16.298403 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
W20240503 02:09:16.298859 27656 leader_election.cc:341] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:16.298889 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8424c39f615456e8db186a9c124629c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.298936 27449 raft_consensus.cc:3037] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.299526 27449 raft_consensus.cc:2454] T d8424c39f615456e8db186a9c124629c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.299685 27386 leader_election.cc:302] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.299739 27524 leader_election.cc:302] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.299785 27659 leader_election.cc:302] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:16.299803 27848 raft_consensus.cc:2781] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.299847 27848 raft_consensus.cc:686] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.299860 27450 raft_consensus.cc:1261] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.299861 28740 raft_consensus.cc:2726] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240503 02:09:16.299880 27848 consensus_queue.cc:237] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.300037 27848 consensus_queue.cc:1035] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.300340 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8424c39f615456e8db186a9c124629c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.300395 27722 raft_consensus.cc:3037] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.300531 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8424c39f615456e8db186a9c124629c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.300590 28685 raft_consensus.cc:2781] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.300637 28685 raft_consensus.cc:686] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.300670 28685 consensus_queue.cc:237] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.300995 27722 raft_consensus.cc:2454] T d8424c39f615456e8db186a9c124629c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.301579 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 62eb4d0cbce545c8ad87883b9be0f6cb (table default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274]). Expected version 1 got 0
I20240503 02:09:16.301623 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
W20240503 02:09:16.301677 27196 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 24e32edb5ccf44199fce2ec6a2c8cf63 consensus state has no leader
I20240503 02:09:16.301926 28656 tablet_bootstrap.cc:492] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.301963 28656 ts_tablet_manager.cc:1367] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:16.302069 28656 raft_consensus.cc:348] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.302111 28656 raft_consensus.cc:374] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.302126 28656 raft_consensus.cc:729] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.302151 28656 consensus_queue.cc:260] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.302229 28656 ts_tablet_manager.cc:1398] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.302294 28656 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.302495 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f1976661e4dba8b366773533ca59e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.302551 27449 raft_consensus.cc:2454] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.302630 28656 tablet_bootstrap.cc:654] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.302670 27659 leader_election.cc:302] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:16.302748 28740 raft_consensus.cc:2781] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.302770 28740 raft_consensus.cc:482] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.302786 28740 raft_consensus.cc:3037] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.302986 29167 tablet.cc:1722] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.303050 28656 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.303084 28656 ts_tablet_manager.cc:1367] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.303186 28656 raft_consensus.cc:348] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.303232 28656 raft_consensus.cc:374] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.303248 28656 raft_consensus.cc:729] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.303285 28656 consensus_queue.cc:260] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.303362 28656 ts_tablet_manager.cc:1398] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.303373 28740 raft_consensus.cc:504] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.303406 28656 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.303473 28740 leader_election.cc:288] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.303730 28656 tablet_bootstrap.cc:654] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.303707 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f1976661e4dba8b366773533ca59e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.303776 27450 raft_consensus.cc:3037] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.303843 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b16f1976661e4dba8b366773533ca59e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.303898 27587 raft_consensus.cc:3037] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.304319 27450 raft_consensus.cc:2454] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.304405 27587 raft_consensus.cc:2454] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.304525 28710 tablet.cc:1722] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.304538 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_cbcba1cbea9d4431aadd54adea024b1e" } modify_external_catalogs: true
I20240503 02:09:16.304587 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_cbcba1cbea9d4431aadd54adea024b1e" } modify_external_catalogs: true
I20240503 02:09:16.304584 27659 leader_election.cc:302] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.304665 28740 raft_consensus.cc:2781] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.304697 28740 raft_consensus.cc:686] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.304730 28740 consensus_queue.cc:237] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.305080 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 62eb4d0cbce545c8ad87883b9be0f6cb (table default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274]). Expected version 1 got 0
I20240503 02:09:16.305145 27196 catalog_manager.cc:5484] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.306077 28694 tablet.cc:1722] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.306524 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet f9f8911e604043c8a5b313ef62d206f0 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.306521 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9ae759f2c210464499b9b30fa848dbcf" } modify_external_catalogs: true
I20240503 02:09:16.306754 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9ae759f2c210464499b9b30fa848dbcf" } modify_external_catalogs: true
I20240503 02:09:16.306751 27198 catalog_manager.cc:5484] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.306970 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
I20240503 02:09:16.307264 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 409fa97790fe47f89b6d95a392e2e334
I20240503 02:09:16.307340 27202 catalog_manager.cc:5484] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.307451 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 409fa97790fe47f89b6d95a392e2e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.307580 28661 tablet_replica.cc:330] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.307628 28661 raft_consensus.cc:2229] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.307658 28661 raft_consensus.cc:2258] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.307811 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 1ab33852711d4212b1603fa04904cf0a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.307888 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 409fa97790fe47f89b6d95a392e2e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.308007 28654 tablet_replica.cc:330] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.308061 28654 raft_consensus.cc:2229] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.308086 28654 raft_consensus.cc:2258] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.308089 28661 ts_tablet_manager.cc:1875] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.308553 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 409fa97790fe47f89b6d95a392e2e334 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.309218 28661 ts_tablet_manager.cc:1888] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.309262 28661 log.cc:1196] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/409fa97790fe47f89b6d95a392e2e334
I20240503 02:09:16.309419 28653 tablet_replica.cc:330] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.309461 28661 ts_tablet_manager.cc:1909] T 409fa97790fe47f89b6d95a392e2e334 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.309473 28653 raft_consensus.cc:2229] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.309499 28653 raft_consensus.cc:2258] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.309628 27853 raft_consensus.cc:482] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.309660 27853 raft_consensus.cc:504] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.309741 27853 leader_election.cc:288] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.309924 28653 ts_tablet_manager.cc:1875] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.310170 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e776dc8757154957900b9fdecbe4dcc1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.310222 27587 raft_consensus.cc:2454] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.310290 28655 tablet_bootstrap.cc:492] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.310328 28655 ts_tablet_manager.cc:1367] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240503 02:09:16.310328 27656 leader_election.cc:302] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.310438 28655 raft_consensus.cc:348] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.310492 28655 raft_consensus.cc:374] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.310480 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e776dc8757154957900b9fdecbe4dcc1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.310544 28655 raft_consensus.cc:729] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.310560 27453 raft_consensus.cc:2454] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.310570 28655 consensus_queue.cc:260] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.310703 27853 raft_consensus.cc:2781] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.310717 28655 ts_tablet_manager.cc:1398] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.310734 27853 raft_consensus.cc:482] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.310751 27853 raft_consensus.cc:3037] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.310768 28655 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.311085 28655 tablet_bootstrap.cc:654] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.311117 28653 ts_tablet_manager.cc:1888] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.311159 28653 log.cc:1196] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/409fa97790fe47f89b6d95a392e2e334
I20240503 02:09:16.311296 28653 ts_tablet_manager.cc:1909] T 409fa97790fe47f89b6d95a392e2e334 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.311269 27853 raft_consensus.cc:504] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.311365 27853 leader_election.cc:288] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.311861 28656 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.311894 28656 ts_tablet_manager.cc:1367] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.312000 28656 raft_consensus.cc:348] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.312054 28656 raft_consensus.cc:374] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.312072 28656 raft_consensus.cc:729] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.312103 28656 consensus_queue.cc:260] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.312145 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f9f8911e604043c8a5b313ef62d206f0
I20240503 02:09:16.312177 28656 ts_tablet_manager.cc:1398] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.312212 27201 catalog_manager.cc:6554] ReplaceTablet: tablet f9f8911e604043c8a5b313ef62d206f0 deleted and replaced by tablet d20aa1acdfbf4c27ba2b5a5dbc75483d
I20240503 02:09:16.312220 28656 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.312423 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e776dc8757154957900b9fdecbe4dcc1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.312458 28654 ts_tablet_manager.cc:1875] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.312482 27453 raft_consensus.cc:3037] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.312546 28656 tablet_bootstrap.cc:654] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.313023 27453 raft_consensus.cc:2454] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.313033 27848 consensus_queue.cc:1035] T 2e3b2192b2b34def9eef4e390822be39 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.313308 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e776dc8757154957900b9fdecbe4dcc1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.313385 27587 raft_consensus.cc:3037] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.313421 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f8911e604043c8a5b313ef62d206f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d20aa1acdfbf4c27ba2b5a5dbc75483d) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.313719 28654 ts_tablet_manager.cc:1888] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.313758 28654 log.cc:1196] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/409fa97790fe47f89b6d95a392e2e334
I20240503 02:09:16.313894 28654 ts_tablet_manager.cc:1909] T 409fa97790fe47f89b6d95a392e2e334 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.313905 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 409fa97790fe47f89b6d95a392e2e334 (table default.Table_cbcba1cbea9d4431aadd54adea024b1e [id=f7d862ad0364458ba75b38483172b371]) successfully deleted
I20240503 02:09:16.313920 27587 raft_consensus.cc:2454] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.314214 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1e86a1d93fb64511aa9def6f04e05b13
I20240503 02:09:16.314229 28654 tablet_replica.cc:330] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.314224 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f8911e604043c8a5b313ef62d206f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d20aa1acdfbf4c27ba2b5a5dbc75483d) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.314278 28654 raft_consensus.cc:2229] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.314298 27656 leader_election.cc:302] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.314328 28654 raft_consensus.cc:2258] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.314352 28653 tablet_replica.cc:330] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.314415 28653 raft_consensus.cc:2229] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.314447 28653 raft_consensus.cc:2258] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.314507 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet f9f8911e604043c8a5b313ef62d206f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d20aa1acdfbf4c27ba2b5a5dbc75483d) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.314633 28653 ts_tablet_manager.cc:1875] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.314729 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 409fa97790fe47f89b6d95a392e2e334 (table default.Table_cbcba1cbea9d4431aadd54adea024b1e [id=f7d862ad0364458ba75b38483172b371]) successfully deleted
I20240503 02:09:16.314880 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.315774 28653 ts_tablet_manager.cc:1888] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.315815 28653 log.cc:1196] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f9f8911e604043c8a5b313ef62d206f0
I20240503 02:09:16.315982 28653 ts_tablet_manager.cc:1909] T f9f8911e604043c8a5b313ef62d206f0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.316612 27692 tablet_service.cc:1467] Processing CreateTablet for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.316699 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.316982 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.317035 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.317189 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.317358 28661 tablet_replica.cc:330] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.317407 28661 raft_consensus.cc:2229] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.317435 28661 raft_consensus.cc:2258] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.317741 28654 ts_tablet_manager.cc:1875] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.317850 28664 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.317858 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 409fa97790fe47f89b6d95a392e2e334 (table default.Table_cbcba1cbea9d4431aadd54adea024b1e [id=f7d862ad0364458ba75b38483172b371]) successfully deleted
I20240503 02:09:16.318176 28664 tablet_bootstrap.cc:654] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.318243 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
I20240503 02:09:16.318295 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.318394 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1ab33852711d4212b1603fa04904cf0a
I20240503 02:09:16.318466 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 1ab33852711d4212b1603fa04904cf0a deleted and replaced by tablet 3c6ff433791d4ab0a00933702258a55a
I20240503 02:09:16.318466 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (DEFAULT_TABLE table=default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.318735 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab33852711d4212b1603fa04904cf0a with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ff433791d4ab0a00933702258a55a) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.318805 28654 ts_tablet_manager.cc:1888] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.318847 28654 log.cc:1196] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f9f8911e604043c8a5b313ef62d206f0
I20240503 02:09:16.318877 28664 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.318913 28664 ts_tablet_manager.cc:1367] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.319032 28654 ts_tablet_manager.cc:1909] T f9f8911e604043c8a5b313ef62d206f0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.319057 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.319033 28664 raft_consensus.cc:348] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.319087 28664 raft_consensus.cc:374] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.319103 28664 raft_consensus.cc:729] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.319126 28664 consensus_queue.cc:260] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.319214 28664 ts_tablet_manager.cc:1398] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.319269 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 203665395f3745f196fb393dd82b8fc9 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.319355 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274] alter complete (version 1)
I20240503 02:09:16.319689 28661 ts_tablet_manager.cc:1875] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.319968 28693 raft_consensus.cc:2781] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.320022 28693 raft_consensus.cc:686] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.320055 28693 consensus_queue.cc:237] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.320427 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6ff433791d4ab0a00933702258a55a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.320506 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6ff433791d4ab0a00933702258a55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.320525 28710 tablet.cc:1722] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.320744 28409 raft_consensus.cc:482] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.320781 28409 raft_consensus.cc:504] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.320844 28661 ts_tablet_manager.cc:1888] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.320859 28409 leader_election.cc:288] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.320894 28661 log.cc:1196] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f9f8911e604043c8a5b313ef62d206f0
I20240503 02:09:16.321065 28661 ts_tablet_manager.cc:1909] T f9f8911e604043c8a5b313ef62d206f0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.321336 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6ff433791d4ab0a00933702258a55a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.321358 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8d8edf43434e5597e58084babb7843" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.321413 27719 raft_consensus.cc:2454] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.321421 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab33852711d4212b1603fa04904cf0a with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ff433791d4ab0a00933702258a55a) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.321525 27388 leader_election.cc:302] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.321559 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e86a1d93fb64511aa9def6f04e05b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.321660 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6ff433791d4ab0a00933702258a55a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.321741 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6ff433791d4ab0a00933702258a55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.321785 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:16.322137 28655 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.322180 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.322193 28655 ts_tablet_manager.cc:1367] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:16.322191 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8d8edf43434e5597e58084babb7843" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.322242 27587 raft_consensus.cc:2454] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:16.322247 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet d20aa1acdfbf4c27ba2b5a5dbc75483d consensus state has no leader
I20240503 02:09:16.322310 28655 raft_consensus.cc:348] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.322371 28655 raft_consensus.cc:374] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.322388 28655 raft_consensus.cc:729] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.322414 28655 consensus_queue.cc:260] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.322490 28655 ts_tablet_manager.cc:1398] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.322607 28766 raft_consensus.cc:2781] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.322635 28766 raft_consensus.cc:482] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.322651 28766 raft_consensus.cc:3037] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.322654 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ab33852711d4212b1603fa04904cf0a with delete_type TABLET_DATA_DELETED (replaced by tablet 3c6ff433791d4ab0a00933702258a55a) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.322993 28664 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.323107 28661 tablet_replica.cc:330] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.323149 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.323160 28661 raft_consensus.cc:2229] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.323186 28661 raft_consensus.cc:2258] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.323189 27197 catalog_manager.cc:5484] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.323241 28766 raft_consensus.cc:504] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.323329 28664 tablet_bootstrap.cc:654] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.323343 28766 leader_election.cc:288] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.323348 28661 ts_tablet_manager.cc:1875] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.323627 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:16.323668 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8d8edf43434e5597e58084babb7843" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.323716 27719 raft_consensus.cc:3037] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.324119 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6ff433791d4ab0a00933702258a55a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.324196 28654 tablet_replica.cc:330] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.324267 28654 raft_consensus.cc:2229] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.324296 28654 raft_consensus.cc:2258] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.324290 27719 raft_consensus.cc:2454] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.324410 28661 ts_tablet_manager.cc:1888] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.324447 28661 log.cc:1196] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1e86a1d93fb64511aa9def6f04e05b13
I20240503 02:09:16.324466 28654 ts_tablet_manager.cc:1875] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.324584 28661 ts_tablet_manager.cc:1909] T 1e86a1d93fb64511aa9def6f04e05b13 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.324792 27388 leader_election.cc:302] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.324856 28766 raft_consensus.cc:2781] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.324887 28766 raft_consensus.cc:686] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.324918 28766 consensus_queue.cc:237] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.325021 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f9f8911e604043c8a5b313ef62d206f0 (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]) successfully deleted
I20240503 02:09:16.325064 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8d8edf43434e5597e58084babb7843" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.325111 27587 raft_consensus.cc:3037] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.325671 27587 raft_consensus.cc:2454] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.326138 28654 ts_tablet_manager.cc:1888] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.326184 28654 log.cc:1196] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1e86a1d93fb64511aa9def6f04e05b13
I20240503 02:09:16.326206 28664 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.326246 28664 ts_tablet_manager.cc:1367] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.326378 28664 raft_consensus.cc:348] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.326426 28664 raft_consensus.cc:374] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.326442 28664 raft_consensus.cc:729] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.326472 28664 consensus_queue.cc:260] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.326565 28654 ts_tablet_manager.cc:1909] T 1e86a1d93fb64511aa9def6f04e05b13 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.326566 28664 ts_tablet_manager.cc:1398] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.326639 28655 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.326967 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1e86a1d93fb64511aa9def6f04e05b13 (table default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]) successfully deleted
I20240503 02:09:16.327040 28654 tablet_replica.cc:330] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.327095 28655 tablet_bootstrap.cc:654] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.327373 28653 tablet_replica.cc:330] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.327430 28653 raft_consensus.cc:2229] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.327495 28653 raft_consensus.cc:2258] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.327598 28655 tablet_bootstrap.cc:492] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.327633 28655 ts_tablet_manager.cc:1367] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.327091 28654 raft_consensus.cc:2229] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.327694 28654 raft_consensus.cc:2258] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.327736 28653 ts_tablet_manager.cc:1875] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.327752 28655 raft_consensus.cc:348] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.327798 28655 raft_consensus.cc:374] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.327816 28655 raft_consensus.cc:729] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.327850 28655 consensus_queue.cc:260] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.327859 28661 tablet_replica.cc:330] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.327908 28661 raft_consensus.cc:2229] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.327940 28661 raft_consensus.cc:2258] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.327940 28655 ts_tablet_manager.cc:1398] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.327984 28655 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.328291 28655 tablet_bootstrap.cc:654] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.328354 28661 ts_tablet_manager.cc:1875] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.328766 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1e86a1d93fb64511aa9def6f04e05b13 (table default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]) successfully deleted
I20240503 02:09:16.328764 28655 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.328823 28655 ts_tablet_manager.cc:1367] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.328883 28653 ts_tablet_manager.cc:1888] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.328920 28653 log.cc:1196] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1ab33852711d4212b1603fa04904cf0a
I20240503 02:09:16.328969 28655 raft_consensus.cc:348] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.329039 28655 raft_consensus.cc:374] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.329057 28655 raft_consensus.cc:729] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.329083 28655 consensus_queue.cc:260] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.329097 28653 ts_tablet_manager.cc:1909] T 1ab33852711d4212b1603fa04904cf0a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.329164 28655 ts_tablet_manager.cc:1398] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.329211 28655 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.329413 28654 ts_tablet_manager.cc:1875] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.329555 28655 tablet_bootstrap.cc:654] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.329679 28653 tablet_replica.cc:330] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.329732 28653 raft_consensus.cc:2229] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.329758 28653 raft_consensus.cc:2258] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.329977 28653 ts_tablet_manager.cc:1875] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.330019 27853 raft_consensus.cc:482] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.330055 27853 raft_consensus.cc:504] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.330137 27853 leader_election.cc:288] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.330492 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
W20240503 02:09:16.330710 27656 leader_election.cc:341] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:16.330828 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.330897 27449 raft_consensus.cc:2454] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.330969 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1ab33852711d4212b1603fa04904cf0a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.331014 28661 ts_tablet_manager.cc:1888] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.331059 28661 log.cc:1196] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1ab33852711d4212b1603fa04904cf0a
I20240503 02:09:16.331061 28653 ts_tablet_manager.cc:1888] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.331097 28653 log.cc:1196] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1e86a1d93fb64511aa9def6f04e05b13
I20240503 02:09:16.331235 28653 ts_tablet_manager.cc:1909] T 1e86a1d93fb64511aa9def6f04e05b13 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.331234 27659 leader_election.cc:302] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:16.331277 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.331315 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.331359 28740 raft_consensus.cc:2781] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.331382 28740 raft_consensus.cc:482] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.331339 27197 catalog_manager.cc:5484] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.331398 28740 raft_consensus.cc:3037] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.331941 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1e86a1d93fb64511aa9def6f04e05b13 (table default.Table_9ae759f2c210464499b9b30fa848dbcf [id=096d883765ab46f8b55ad54853b3e449]) successfully deleted
I20240503 02:09:16.331976 28740 raft_consensus.cc:504] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.332022 28654 ts_tablet_manager.cc:1888] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.332058 28654 log.cc:1196] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1ab33852711d4212b1603fa04904cf0a
I20240503 02:09:16.332096 28740 leader_election.cc:288] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.332384 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.332468 28661 ts_tablet_manager.cc:1909] T 1ab33852711d4212b1603fa04904cf0a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
W20240503 02:09:16.332592 27656 leader_election.cc:341] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:16.332726 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1ab33852711d4212b1603fa04904cf0a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.332821 28654 ts_tablet_manager.cc:1909] T 1ab33852711d4212b1603fa04904cf0a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.333060 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1ab33852711d4212b1603fa04904cf0a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.333223 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.333278 27449 raft_consensus.cc:3037] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.333827 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.333603 28655 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.333997 28655 ts_tablet_manager.cc:1367] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:16.334061 27449 raft_consensus.cc:2454] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.334123 28655 raft_consensus.cc:348] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.334158 28655 raft_consensus.cc:374] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.334168 28655 raft_consensus.cc:729] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.334213 28655 consensus_queue.cc:260] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.334270 27659 leader_election.cc:302] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:16.334353 28740 raft_consensus.cc:2781] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.334384 28740 raft_consensus.cc:686] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.334414 28740 consensus_queue.cc:237] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.334841 28656 tablet_bootstrap.cc:492] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.334877 28656 ts_tablet_manager.cc:1367] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.000s
I20240503 02:09:16.335064 28656 raft_consensus.cc:348] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.335114 28656 raft_consensus.cc:374] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.335130 28656 raft_consensus.cc:729] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.335121 28655 ts_tablet_manager.cc:1398] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.335160 28656 consensus_queue.cc:260] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.335233 28656 ts_tablet_manager.cc:1398] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.335314 28656 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.335630 28656 tablet_bootstrap.cc:654] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.336177 27202 catalog_manager.cc:5484] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.336493 28409 raft_consensus.cc:482] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.336536 28409 raft_consensus.cc:504] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.336620 28409 leader_election.cc:288] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.336776 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3eb52c8c0a94673bf1546f71320a6fe" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.336831 27723 raft_consensus.cc:2454] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.337088 27388 leader_election.cc:302] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.337154 28409 raft_consensus.cc:2781] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.337184 28409 raft_consensus.cc:482] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.337199 28409 raft_consensus.cc:3037] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.337579 28656 tablet_bootstrap.cc:492] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.337616 28656 ts_tablet_manager.cc:1367] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.337724 28656 raft_consensus.cc:348] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.337770 28656 raft_consensus.cc:374] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.337788 28656 raft_consensus.cc:729] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.337771 28409 raft_consensus.cc:504] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.337826 28656 consensus_queue.cc:260] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.337872 28409 leader_election.cc:288] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.337895 28656 ts_tablet_manager.cc:1398] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.337949 28656 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.338027 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3eb52c8c0a94673bf1546f71320a6fe" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.338032 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3eb52c8c0a94673bf1546f71320a6fe" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.338089 27587 raft_consensus.cc:2454] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.338182 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3eb52c8c0a94673bf1546f71320a6fe" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.338232 27723 raft_consensus.cc:3037] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.338246 28656 tablet_bootstrap.cc:654] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.338689 28656 tablet_bootstrap.cc:492] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.338723 28656 ts_tablet_manager.cc:1367] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.338830 27723 raft_consensus.cc:2454] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.338833 28656 raft_consensus.cc:348] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.338882 28656 raft_consensus.cc:374] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.338900 28656 raft_consensus.cc:729] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.338924 28656 consensus_queue.cc:260] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.339094 28656 ts_tablet_manager.cc:1398] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.339576 27388 leader_election.cc:302] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:16.339704 28409 raft_consensus.cc:2781] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.339752 28409 raft_consensus.cc:686] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.339784 28409 consensus_queue.cc:237] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.339813 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.340783 27196 catalog_manager.cc:5484] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.343607 27582 raft_consensus.cc:1261] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.343652 27449 raft_consensus.cc:1261] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.343822 28740 consensus_queue.cc:1035] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.343904 28740 consensus_queue.cc:1035] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.344897 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
I20240503 02:09:16.345238 28694 tablet.cc:1722] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.345530 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]). Expected version 1 got 0
I20240503 02:09:16.345901 29167 tablet.cc:1722] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.346438 28710 tablet.cc:1722] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.348660 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d] alter complete (version 1)
I20240503 02:09:16.377422 27848 raft_consensus.cc:482] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.377473 27848 raft_consensus.cc:504] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.377583 27848 leader_election.cc:288] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.377725 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f7c3e3e6e643948b8668c675a20b89" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.377779 27449 raft_consensus.cc:2454] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.377746 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f7c3e3e6e643948b8668c675a20b89" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.377818 27723 raft_consensus.cc:2454] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.377907 27524 leader_election.cc:302] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.377995 27848 raft_consensus.cc:2781] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.378036 27848 raft_consensus.cc:482] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.378062 27848 raft_consensus.cc:3037] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.378523 27848 raft_consensus.cc:504] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.378636 27848 leader_election.cc:288] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.378758 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f7c3e3e6e643948b8668c675a20b89" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.378775 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36f7c3e3e6e643948b8668c675a20b89" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.378816 27449 raft_consensus.cc:3037] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.378834 27723 raft_consensus.cc:3037] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.379261 27723 raft_consensus.cc:2454] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.379266 27449 raft_consensus.cc:2454] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.379405 27524 leader_election.cc:302] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.379499 27848 raft_consensus.cc:2781] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.379554 27848 raft_consensus.cc:686] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.379598 27848 consensus_queue.cc:237] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.380297 27197 catalog_manager.cc:5484] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.383177 27848 consensus_queue.cc:1035] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.384665 28410 consensus_queue.cc:1035] T 3630cebdb1b34cebab8118c250543ade P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.388926 27853 consensus_queue.cc:1035] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.390430 28693 consensus_queue.cc:1035] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.401626 28410 consensus_queue.cc:1035] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.402559 28410 consensus_queue.cc:1035] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.404337 28409 raft_consensus.cc:482] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.404394 28409 raft_consensus.cc:504] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.404501 28409 leader_election.cc:288] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.404623 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f852b718f524467947b42f653f3530b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.404619 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f852b718f524467947b42f653f3530b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.404688 27723 raft_consensus.cc:2454] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.404688 27582 raft_consensus.cc:2454] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.404850 27386 leader_election.cc:302] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.404930 28409 raft_consensus.cc:2781] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.404973 28409 raft_consensus.cc:482] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.404997 28409 raft_consensus.cc:3037] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.405470 28409 raft_consensus.cc:504] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.405567 28409 leader_election.cc:288] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.405699 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f852b718f524467947b42f653f3530b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.405710 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f852b718f524467947b42f653f3530b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.405767 27582 raft_consensus.cc:3037] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.405766 27723 raft_consensus.cc:3037] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.406208 27723 raft_consensus.cc:2454] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.406239 27582 raft_consensus.cc:2454] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.406373 27386 leader_election.cc:302] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.406452 28409 raft_consensus.cc:2781] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.406503 28409 raft_consensus.cc:686] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.406549 28409 consensus_queue.cc:237] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.407186 27197 catalog_manager.cc:5484] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.407635 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f553462ad4ea48fdb713726340f763d4" } modify_external_catalogs: true
I20240503 02:09:16.407701 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f553462ad4ea48fdb713726340f763d4" } modify_external_catalogs: true
I20240503 02:09:16.408972 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 06be437ed5114166b802559170e440eb
I20240503 02:09:16.409144 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 06be437ed5114166b802559170e440eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.409217 28653 tablet_replica.cc:330] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.409229 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 06be437ed5114166b802559170e440eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.409233 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 06be437ed5114166b802559170e440eb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.409266 28653 raft_consensus.cc:2229] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.409296 28653 raft_consensus.cc:2258] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.409312 28661 tablet_replica.cc:330] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.409377 28661 raft_consensus.cc:2229] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.409405 28661 raft_consensus.cc:2258] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.409440 28653 ts_tablet_manager.cc:1875] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.409493 28661 ts_tablet_manager.cc:1875] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.410662 28654 tablet_replica.cc:330] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.410714 28653 ts_tablet_manager.cc:1888] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.410725 28654 raft_consensus.cc:2229] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.410748 28653 log.cc:1196] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/06be437ed5114166b802559170e440eb
I20240503 02:09:16.410768 28654 pending_rounds.cc:62] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:16.410787 28654 pending_rounds.cc:68] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420031538012160 op_type: NO_OP noop_request { }
I20240503 02:09:16.410815 28654 raft_consensus.cc:2866] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.410835 28654 raft_consensus.cc:2258] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.410861 28653 ts_tablet_manager.cc:1909] T 06be437ed5114166b802559170e440eb P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.411072 28654 ts_tablet_manager.cc:1875] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.411206 28661 ts_tablet_manager.cc:1888] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.411247 28661 log.cc:1196] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/06be437ed5114166b802559170e440eb
I20240503 02:09:16.411366 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 06be437ed5114166b802559170e440eb (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:16.411417 28661 ts_tablet_manager.cc:1909] T 06be437ed5114166b802559170e440eb P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.411669 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 06be437ed5114166b802559170e440eb (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:16.412127 28654 ts_tablet_manager.cc:1888] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.412168 28654 log.cc:1196] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/06be437ed5114166b802559170e440eb
I20240503 02:09:16.412315 28654 ts_tablet_manager.cc:1909] T 06be437ed5114166b802559170e440eb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.412567 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 06be437ed5114166b802559170e440eb (table default.Table_f553462ad4ea48fdb713726340f763d4 [id=c6ea1ca016fc4e8f8d494d1dc0bc40e2]) successfully deleted
I20240503 02:09:16.424819 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_98eb120ca20040b89e4cfbdd3e5a75bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.424939 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98eb120ca20040b89e4cfbdd3e5a75bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.426834 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 58740c221e9b422d9fcff96e2df8e85e (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.426828 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 58740c221e9b422d9fcff96e2df8e85e (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.426929 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58740c221e9b422d9fcff96e2df8e85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.426930 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58740c221e9b422d9fcff96e2df8e85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.427275 28409 raft_consensus.cc:482] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.427330 28409 raft_consensus.cc:504] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.427424 28409 leader_election.cc:288] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.427546 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89add66b30d49348552022b8f5f8ce3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.427605 27582 raft_consensus.cc:2454] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.427718 27386 leader_election.cc:302] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.427721 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89add66b30d49348552022b8f5f8ce3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.427772 27723 raft_consensus.cc:2454] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.427821 28409 raft_consensus.cc:2781] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.427856 28409 raft_consensus.cc:482] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.427872 28409 raft_consensus.cc:3037] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.428022 28664 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.428375 28664 tablet_bootstrap.cc:654] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.428411 28409 raft_consensus.cc:504] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.428532 28409 leader_election.cc:288] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.428612 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 58740c221e9b422d9fcff96e2df8e85e (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.428692 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58740c221e9b422d9fcff96e2df8e85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.428757 28655 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.428941 28664 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.428972 28664 ts_tablet_manager.cc:1367] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.429075 28664 raft_consensus.cc:348] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.429088 28655 tablet_bootstrap.cc:654] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.429108 28664 raft_consensus.cc:374] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.429121 28664 raft_consensus.cc:729] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.429154 28664 consensus_queue.cc:260] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.429242 28664 ts_tablet_manager.cc:1398] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.429847 28656 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.430191 28656 tablet_bootstrap.cc:654] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.430344 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89add66b30d49348552022b8f5f8ce3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.430414 27723 raft_consensus.cc:3037] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.430557 28655 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.430591 28655 ts_tablet_manager.cc:1367] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.430567 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_49dc43dbcbba437e8e7a442d4cda81b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.430648 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49dc43dbcbba437e8e7a442d4cda81b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.430706 28655 raft_consensus.cc:348] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.430752 28655 raft_consensus.cc:374] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.430790 28655 raft_consensus.cc:729] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.430825 28655 consensus_queue.cc:260] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.430923 28655 ts_tablet_manager.cc:1398] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.430960 27723 raft_consensus.cc:2454] T d89add66b30d49348552022b8f5f8ce3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.430972 28656 tablet_bootstrap.cc:492] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.431011 28656 ts_tablet_manager.cc:1367] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.431125 28656 raft_consensus.cc:348] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.431172 28656 raft_consensus.cc:374] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.431188 28656 raft_consensus.cc:729] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.431224 28656 consensus_queue.cc:260] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.431310 28656 ts_tablet_manager.cc:1398] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.431478 27388 leader_election.cc:302] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.431540 28409 raft_consensus.cc:2781] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.431568 28409 raft_consensus.cc:686] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.431599 28409 consensus_queue.cc:237] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.432482 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89add66b30d49348552022b8f5f8ce3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.432533 27582 raft_consensus.cc:3037] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.433099 27582 raft_consensus.cc:2454] T d89add66b30d49348552022b8f5f8ce3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.434504 27196 catalog_manager.cc:5484] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.434589 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_603bc38c1af74b51a2983f8d604e5eb5" } modify_external_catalogs: true
I20240503 02:09:16.434610 27427 tablet_service.cc:1467] Processing CreateTablet for tablet a1177866c9e94ce9bfdc45ecbc6172d2 (DEFAULT_TABLE table=default.Table_49dc43dbcbba437e8e7a442d4cda81b3 [id=89289d6ec12b4857ab4449223dc00d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.434648 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_603bc38c1af74b51a2983f8d604e5eb5" } modify_external_catalogs: true
I20240503 02:09:16.434690 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1177866c9e94ce9bfdc45ecbc6172d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.434685 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a1177866c9e94ce9bfdc45ecbc6172d2 (DEFAULT_TABLE table=default.Table_49dc43dbcbba437e8e7a442d4cda81b3 [id=89289d6ec12b4857ab4449223dc00d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.434712 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a1177866c9e94ce9bfdc45ecbc6172d2 (DEFAULT_TABLE table=default.Table_49dc43dbcbba437e8e7a442d4cda81b3 [id=89289d6ec12b4857ab4449223dc00d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.434777 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1177866c9e94ce9bfdc45ecbc6172d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.434792 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1177866c9e94ce9bfdc45ecbc6172d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.435499 28656 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.435719 28664 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.435823 28656 tablet_bootstrap.cc:654] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.436060 28664 tablet_bootstrap.cc:654] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.436228 28655 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.436573 28655 tablet_bootstrap.cc:654] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.436666 28656 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.436709 28656 ts_tablet_manager.cc:1367] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.436844 28664 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.436825 28656 raft_consensus.cc:348] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.436872 28656 raft_consensus.cc:374] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.436882 28664 ts_tablet_manager.cc:1367] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.436889 28656 raft_consensus.cc:729] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.436937 28656 consensus_queue.cc:260] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.437001 28664 raft_consensus.cc:348] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.437031 28656 ts_tablet_manager.cc:1398] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.437054 28664 raft_consensus.cc:374] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.437072 28664 raft_consensus.cc:729] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.437105 28664 consensus_queue.cc:260] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.437172 28655 tablet_bootstrap.cc:492] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.437183 28664 ts_tablet_manager.cc:1398] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.437211 28655 ts_tablet_manager.cc:1367] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.437376 28655 raft_consensus.cc:348] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.437427 28655 raft_consensus.cc:374] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.437444 28655 raft_consensus.cc:729] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.437479 28655 consensus_queue.cc:260] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.437559 28655 ts_tablet_manager.cc:1398] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.438232 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 62eb4d0cbce545c8ad87883b9be0f6cb
I20240503 02:09:16.438650 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.438715 28661 tablet_replica.cc:330] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.438752 28661 raft_consensus.cc:2229] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.438783 28661 raft_consensus.cc:2258] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.439039 28661 ts_tablet_manager.cc:1875] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.439210 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.439278 28654 tablet_replica.cc:330] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.439323 28654 raft_consensus.cc:2229] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.439368 28654 raft_consensus.cc:2258] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.439514 28654 ts_tablet_manager.cc:1875] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.439797 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 62eb4d0cbce545c8ad87883b9be0f6cb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.440258 28661 ts_tablet_manager.cc:1888] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.440301 28661 log.cc:1196] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/62eb4d0cbce545c8ad87883b9be0f6cb
I20240503 02:09:16.440481 28661 ts_tablet_manager.cc:1909] T 62eb4d0cbce545c8ad87883b9be0f6cb P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.440506 28654 ts_tablet_manager.cc:1888] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.440538 28654 log.cc:1196] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/62eb4d0cbce545c8ad87883b9be0f6cb
I20240503 02:09:16.440698 28654 ts_tablet_manager.cc:1909] T 62eb4d0cbce545c8ad87883b9be0f6cb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.440758 28653 tablet_replica.cc:330] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.440855 28653 raft_consensus.cc:2229] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.440893 28653 raft_consensus.cc:2258] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.441030 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_430cab1f60494f67984d34f207740a2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.441056 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 62eb4d0cbce545c8ad87883b9be0f6cb (table default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274]) successfully deleted
W20240503 02:09:16.441103 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_430cab1f60494f67984d34f207740a2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.441529 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_c148f7646e0c499787482efa0d32cb21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.441588 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 62eb4d0cbce545c8ad87883b9be0f6cb (table default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274]) successfully deleted
W20240503 02:09:16.441622 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c148f7646e0c499787482efa0d32cb21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.441812 28653 ts_tablet_manager.cc:1875] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.443027 28653 ts_tablet_manager.cc:1888] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.443066 28653 log.cc:1196] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/62eb4d0cbce545c8ad87883b9be0f6cb
I20240503 02:09:16.443233 28653 ts_tablet_manager.cc:1909] T 62eb4d0cbce545c8ad87883b9be0f6cb P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.444180 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 (DEFAULT_TABLE table=default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.444224 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 (DEFAULT_TABLE table=default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.444267 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b7bf524ad049dba8cd61f4cde3cec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.444293 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b7bf524ad049dba8cd61f4cde3cec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.444658 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 (DEFAULT_TABLE table=default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.444730 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44b7bf524ad049dba8cd61f4cde3cec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.445257 28655 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.445501 28656 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.445621 28655 tablet_bootstrap.cc:654] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.445796 28656 tablet_bootstrap.cc:654] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.446148 28664 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
W20240503 02:09:16.446162 27196 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:16.446520 28664 tablet_bootstrap.cc:654] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.446707 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 9654b6fa66a84fffb440477a95a0f3a9 (DEFAULT_TABLE table=default.Table_c148f7646e0c499787482efa0d32cb21 [id=bf423777e8d14b688a677e37aab7a477]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.446758 28656 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.446784 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9654b6fa66a84fffb440477a95a0f3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.446805 28656 ts_tablet_manager.cc:1367] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.446920 28656 raft_consensus.cc:348] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.446942 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 9654b6fa66a84fffb440477a95a0f3a9 (DEFAULT_TABLE table=default.Table_c148f7646e0c499787482efa0d32cb21 [id=bf423777e8d14b688a677e37aab7a477]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.446970 28656 raft_consensus.cc:374] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.446990 28656 raft_consensus.cc:729] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.447016 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9654b6fa66a84fffb440477a95a0f3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.447026 28656 consensus_queue.cc:260] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.447101 28656 ts_tablet_manager.cc:1398] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.447467 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 9654b6fa66a84fffb440477a95a0f3a9 (DEFAULT_TABLE table=default.Table_c148f7646e0c499787482efa0d32cb21 [id=bf423777e8d14b688a677e37aab7a477]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.447566 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9654b6fa66a84fffb440477a95a0f3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.448443 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 62eb4d0cbce545c8ad87883b9be0f6cb (table default.Table_603bc38c1af74b51a2983f8d604e5eb5 [id=db75dd5ce72f4d18bf7363df6ff19274]) successfully deleted
I20240503 02:09:16.448599 28664 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.448640 28664 ts_tablet_manager.cc:1367] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.448746 28664 raft_consensus.cc:348] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.448779 28664 raft_consensus.cc:374] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.448791 28664 raft_consensus.cc:729] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.448825 28664 consensus_queue.cc:260] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.448871 28656 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.448901 28664 ts_tablet_manager.cc:1398] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.448971 28664 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.449205 28656 tablet_bootstrap.cc:654] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.449293 28664 tablet_bootstrap.cc:654] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.449599 27092 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskpsd1k2/build/release/bin/kudu with pid 28542
I20240503 02:09:16.450948 28664 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.450990 28664 ts_tablet_manager.cc:1367] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.451107 28664 raft_consensus.cc:348] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.451153 28664 raft_consensus.cc:374] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.451169 28664 raft_consensus.cc:729] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.451207 28664 consensus_queue.cc:260] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.451310 28664 ts_tablet_manager.cc:1398] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.451668 28656 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.451719 28656 ts_tablet_manager.cc:1367] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.451766 28655 tablet_bootstrap.cc:492] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.451818 28655 ts_tablet_manager.cc:1367] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:16.451828 28656 raft_consensus.cc:348] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.451892 28656 raft_consensus.cc:374] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.451920 28656 raft_consensus.cc:729] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.451932 28655 raft_consensus.cc:348] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.451982 28655 raft_consensus.cc:374] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.452005 28655 raft_consensus.cc:729] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.451961 28656 consensus_queue.cc:260] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.452039 28655 consensus_queue.cc:260] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.452085 28656 ts_tablet_manager.cc:1398] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.452108 28655 ts_tablet_manager.cc:1398] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.452162 28655 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.452466 28655 tablet_bootstrap.cc:654] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.453251 28655 tablet_bootstrap.cc:492] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.453289 28655 ts_tablet_manager.cc:1367] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.453428 28655 raft_consensus.cc:348] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.453473 28655 raft_consensus.cc:374] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.453490 28655 raft_consensus.cc:729] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.453524 28655 consensus_queue.cc:260] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.453614 28655 ts_tablet_manager.cc:1398] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:16.456250 27522 connection.cc:535] client connection to 127.26.117.62:37529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240503 02:09:16.456305 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0a20b196ff32424d97adbd7278fb3760"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.456393 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a20b196ff32424d97adbd7278fb3760 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:16.456380 27657 connection.cc:535] client connection to 127.26.117.62:37529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:09:16.456597 27387 connection.cc:535] client connection to 127.26.117.62:37529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:09:16.457051 29470 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 02:09:16.456766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.117.62:37529 (local address 127.26.117.2:42949)
0503 02:09:16.456867 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0503 02:09:16.456903 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
W20240503 02:09:16.457674 29469 negotiation.cc:337] Failed RPC negotiation. Trace:
0503 02:09:16.456624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.117.62:37529 (local address 127.26.117.3:34241)
0503 02:09:16.457188 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0503 02:09:16.457238 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":42,"threads_started":1}
W20240503 02:09:16.457813 27188 consensus_peers.cc:485] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 -> Peer c324c9220fd14d3c817b18bbd3774439 (127.26.117.62:37529): Couldn't send request to peer c324c9220fd14d3c817b18bbd3774439. Status: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:16.458659 27427 tablet_service.cc:1467] Processing CreateTablet for tablet b832fd89187a4f9c8e30d646c0541986 (DEFAULT_TABLE table=default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.458750 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b832fd89187a4f9c8e30d646c0541986. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.458743 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b832fd89187a4f9c8e30d646c0541986 (DEFAULT_TABLE table=default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.458753 27565 tablet_service.cc:1467] Processing CreateTablet for tablet b832fd89187a4f9c8e30d646c0541986 (DEFAULT_TABLE table=default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.458822 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b832fd89187a4f9c8e30d646c0541986. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.458817 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b832fd89187a4f9c8e30d646c0541986. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.459475 28656 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.459750 28655 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.459811 28656 tablet_bootstrap.cc:654] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.460163 28655 tablet_bootstrap.cc:654] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.460394 28656 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.460427 28656 ts_tablet_manager.cc:1367] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.460423 28664 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.460549 28656 raft_consensus.cc:348] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.460597 28656 raft_consensus.cc:374] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.460589 28655 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.460615 28656 raft_consensus.cc:729] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.460625 28655 ts_tablet_manager.cc:1367] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.460657 28656 consensus_queue.cc:260] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.460736 28656 ts_tablet_manager.cc:1398] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.460745 28655 raft_consensus.cc:348] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.460778 28664 tablet_bootstrap.cc:654] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.460789 28655 raft_consensus.cc:374] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.460806 28655 raft_consensus.cc:729] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.460842 28655 consensus_queue.cc:260] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.460938 28655 ts_tablet_manager.cc:1398] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.461383 28664 tablet_bootstrap.cc:492] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.461421 28664 ts_tablet_manager.cc:1367] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.461560 28664 raft_consensus.cc:348] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.461611 28664 raft_consensus.cc:374] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.461628 28664 raft_consensus.cc:729] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.461660 28664 consensus_queue.cc:260] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.461745 28664 ts_tablet_manager.cc:1398] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.461905 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8f3497b3f2fd46abb20e2efdefdefb34" } new_table_name: "default.Table_d27da7c2acc24478a93b726989a2d2b9" modify_external_catalogs: true
I20240503 02:09:16.462215 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2528d81d929443d18798a4a22223eb91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.462309 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2528d81d929443d18798a4a22223eb91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.462431 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e5d5a0ac57364d878cf7a738b8a5727b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.462502 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5d5a0ac57364d878cf7a738b8a5727b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.464629 29167 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.464771 28694 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.465852 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_4926b749765c4232b2834d5419cc1ab1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.465946 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4926b749765c4232b2834d5419cc1ab1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.465862 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.466109 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.466562 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 93ad86edb8f646f5b717463b4fae2263 (DEFAULT_TABLE table=default.Table_e5d5a0ac57364d878cf7a738b8a5727b [id=8a1f481254ea4ac696ef9e3e3e477eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.466673 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ad86edb8f646f5b717463b4fae2263. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.466869 27560 tablet_service.cc:1467] Processing CreateTablet for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9 (DEFAULT_TABLE table=default.Table_2528d81d929443d18798a4a22223eb91 [id=b2a2642550f04315bd73cb84d737f49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.466905 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 93ad86edb8f646f5b717463b4fae2263 (DEFAULT_TABLE table=default.Table_e5d5a0ac57364d878cf7a738b8a5727b [id=8a1f481254ea4ac696ef9e3e3e477eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.466965 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.467018 28710 tablet.cc:1722] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.467308 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ad86edb8f646f5b717463b4fae2263. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.467654 27426 tablet_service.cc:1467] Processing CreateTablet for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9 (DEFAULT_TABLE table=default.Table_2528d81d929443d18798a4a22223eb91 [id=b2a2642550f04315bd73cb84d737f49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.467731 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.468170 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 93ad86edb8f646f5b717463b4fae2263 (DEFAULT_TABLE table=default.Table_e5d5a0ac57364d878cf7a738b8a5727b [id=8a1f481254ea4ac696ef9e3e3e477eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.468246 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ad86edb8f646f5b717463b4fae2263. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.468286 28656 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.468616 28656 tablet_bootstrap.cc:654] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.466567 27692 tablet_service.cc:1467] Processing CreateTablet for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9 (DEFAULT_TABLE table=default.Table_2528d81d929443d18798a4a22223eb91 [id=b2a2642550f04315bd73cb84d737f49d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.468837 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcad4e4abb9a47ac9a6c481e91eb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.468924 28664 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.469214 28766 raft_consensus.cc:482] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.469254 28766 raft_consensus.cc:504] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.469281 28664 tablet_bootstrap.cc:654] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.469369 28766 leader_election.cc:288] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.469839 28656 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.469877 28656 ts_tablet_manager.cc:1367] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.469852 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6ff433791d4ab0a00933702258a55a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.469904 28664 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.469919 27723 raft_consensus.cc:2454] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.469938 28664 ts_tablet_manager.cc:1367] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.469988 28656 raft_consensus.cc:348] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.470042 28656 raft_consensus.cc:374] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.470072 28656 raft_consensus.cc:729] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.470105 28656 consensus_queue.cc:260] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.470098 28664 raft_consensus.cc:348] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.470144 28664 raft_consensus.cc:374] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.470161 28664 raft_consensus.cc:729] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.470185 28664 consensus_queue.cc:260] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.470196 28656 ts_tablet_manager.cc:1398] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.470242 28656 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.470273 28664 ts_tablet_manager.cc:1398] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.470288 27388 leader_election.cc:302] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.470341 28664 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.470357 28766 raft_consensus.cc:2781] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.470381 28766 raft_consensus.cc:482] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.470395 28766 raft_consensus.cc:3037] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.470432 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d27da7c2acc24478a93b726989a2d2b9" } modify_external_catalogs: true
I20240503 02:09:16.470482 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d27da7c2acc24478a93b726989a2d2b9" } modify_external_catalogs: true
I20240503 02:09:16.470538 28656 tablet_bootstrap.cc:654] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.470649 28664 tablet_bootstrap.cc:654] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.470834 28655 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.470044 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6ff433791d4ab0a00933702258a55a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.470899 27582 raft_consensus.cc:2454] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.470968 28766 raft_consensus.cc:504] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.471071 28766 leader_election.cc:288] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.471180 28655 tablet_bootstrap.cc:654] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.471246 28740 raft_consensus.cc:482] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.471287 28740 raft_consensus.cc:504] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.471378 28740 leader_election.cc:288] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.471488 28664 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.471524 28664 ts_tablet_manager.cc:1367] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.471530 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e659f5818744f584e64997789ebf86" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.471582 27449 raft_consensus.cc:2454] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.471655 28664 raft_consensus.cc:348] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.471704 28664 raft_consensus.cc:374] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.471676 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e659f5818744f584e64997789ebf86" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.471721 28664 raft_consensus.cc:729] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.471732 27582 raft_consensus.cc:2454] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.471738 27659 leader_election.cc:302] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.471743 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6ff433791d4ab0a00933702258a55a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.471755 28664 consensus_queue.cc:260] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.471784 27587 raft_consensus.cc:3037] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.471843 28664 ts_tablet_manager.cc:1398] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.472039 28655 tablet_bootstrap.cc:492] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.472080 28655 ts_tablet_manager.cc:1367] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.472206 28655 raft_consensus.cc:348] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.472254 28655 raft_consensus.cc:374] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.472272 28655 raft_consensus.cc:729] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.472316 28655 consensus_queue.cc:260] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.472373 27587 raft_consensus.cc:2454] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.472427 28655 ts_tablet_manager.cc:1398] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.472436 27853 raft_consensus.cc:2781] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.472467 27853 raft_consensus.cc:482] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.472484 27853 raft_consensus.cc:3037] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.472519 28655 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.472815 28655 tablet_bootstrap.cc:654] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.472980 27853 raft_consensus.cc:504] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.473099 27853 leader_election.cc:288] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.473242 28655 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.473287 28655 ts_tablet_manager.cc:1367] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.473297 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e659f5818744f584e64997789ebf86" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.473462 27453 raft_consensus.cc:3037] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.471505 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6ff433791d4ab0a00933702258a55a" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.473510 27723 raft_consensus.cc:3037] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.473541 28655 raft_consensus.cc:348] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.473598 28655 raft_consensus.cc:374] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.473619 28655 raft_consensus.cc:729] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.473697 28655 consensus_queue.cc:260] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.473778 28655 ts_tablet_manager.cc:1398] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.474015 27723 raft_consensus.cc:2454] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.474047 27453 raft_consensus.cc:2454] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.474144 27388 leader_election.cc:302] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.474187 27659 leader_election.cc:302] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.474265 28740 raft_consensus.cc:2781] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.474277 28766 raft_consensus.cc:2781] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.474306 28740 raft_consensus.cc:686] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.474325 28766 raft_consensus.cc:686] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.474316 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82e659f5818744f584e64997789ebf86" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.474340 28740 consensus_queue.cc:237] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.474359 28766 consensus_queue.cc:237] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.474370 27587 raft_consensus.cc:3037] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.474951 27587 raft_consensus.cc:2454] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.475749 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a543dbd910a44984910275944aa9e670" } new_table_name: "default.Table_6dc41f3a71694c1e854ddc70851cd997" modify_external_catalogs: true
I20240503 02:09:16.476230 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d27da7c2acc24478a93b726989a2d2b9 [id=7a32c89ef97847cca4b48e7891d95a89] alter complete (version 2)
I20240503 02:09:16.476717 28656 tablet_bootstrap.cc:492] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.476760 28656 ts_tablet_manager.cc:1367] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.476878 28656 raft_consensus.cc:348] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.476922 28656 raft_consensus.cc:374] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.476933 28656 raft_consensus.cc:729] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.476963 28656 consensus_queue.cc:260] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.477041 28656 ts_tablet_manager.cc:1398] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.477384 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 78e562fa324448f788adaa4f60da3076 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.477509 27692 tablet_service.cc:1467] Processing CreateTablet for tablet a710cef4241b47d788dced187487c7cd (DEFAULT_TABLE table=default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.477602 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a710cef4241b47d788dced187487c7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.477835 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a710cef4241b47d788dced187487c7cd (DEFAULT_TABLE table=default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.477916 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a710cef4241b47d788dced187487c7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.478389 28664 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.478653 27427 tablet_service.cc:1467] Processing CreateTablet for tablet a710cef4241b47d788dced187487c7cd (DEFAULT_TABLE table=default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.478744 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a710cef4241b47d788dced187487c7cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.478818 28664 tablet_bootstrap.cc:654] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.479341 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 09426f1e3db54724a043f24c40f0e9f1
I20240503 02:09:16.479676 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.479777 28654 tablet_replica.cc:330] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.479836 28654 raft_consensus.cc:2229] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.479902 28654 raft_consensus.cc:2258] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.480077 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 78e562fa324448f788adaa4f60da3076
I20240503 02:09:16.480154 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 78e562fa324448f788adaa4f60da3076 deleted and replaced by tablet b02d8134f2684fb5a0ddbb9e685c8f79
I20240503 02:09:16.480419 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e562fa324448f788adaa4f60da3076 with delete_type TABLET_DATA_DELETED (replaced by tablet b02d8134f2684fb5a0ddbb9e685c8f79) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.480473 28654 ts_tablet_manager.cc:1875] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.480486 28661 tablet_replica.cc:330] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.480525 28661 raft_consensus.cc:2229] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.480554 28661 raft_consensus.cc:2258] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.480813 28661 ts_tablet_manager.cc:1875] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.480813 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.480871 28655 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.481444 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 (DEFAULT_TABLE table=default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3 [id=56970436dd49486f86fc2fc41f5de5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.481494 28655 tablet_bootstrap.cc:654] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.481529 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010d95e45f6e417c95d4d3e6278f4d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.481738 28654 ts_tablet_manager.cc:1888] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:16.481775 28654 log.cc:1196] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/09426f1e3db54724a043f24c40f0e9f1
I20240503 02:09:16.481846 28507 raft_consensus.cc:482] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.481889 28507 raft_consensus.cc:504] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.481953 28654 ts_tablet_manager.cc:1909] T 09426f1e3db54724a043f24c40f0e9f1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.481979 28507 leader_election.cc:288] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.482116 28661 ts_tablet_manager.cc:1888] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.482156 28661 log.cc:1196] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/78e562fa324448f788adaa4f60da3076
I20240503 02:09:16.482237 28655 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.482268 28655 ts_tablet_manager.cc:1367] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.482272 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_d27da7c2acc24478a93b726989a2d2b9 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
I20240503 02:09:16.482329 28661 ts_tablet_manager.cc:1909] T 78e562fa324448f788adaa4f60da3076 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.482414 28655 raft_consensus.cc:348] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.482482 28655 raft_consensus.cc:374] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.482504 28655 raft_consensus.cc:729] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.482482 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58740c221e9b422d9fcff96e2df8e85e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.482540 27723 raft_consensus.cc:2454] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.482542 28655 consensus_queue.cc:260] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.482623 28655 ts_tablet_manager.cc:1398] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.482681 27523 leader_election.cc:302] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.482750 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58740c221e9b422d9fcff96e2df8e85e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.482759 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e562fa324448f788adaa4f60da3076 with delete_type TABLET_DATA_DELETED (replaced by tablet b02d8134f2684fb5a0ddbb9e685c8f79) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.482820 27449 raft_consensus.cc:2454] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.482847 28653 tablet_replica.cc:330] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.482892 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 (DEFAULT_TABLE table=default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3 [id=56970436dd49486f86fc2fc41f5de5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.482896 28653 raft_consensus.cc:2229] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.482990 28653 raft_consensus.cc:2258] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.483000 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010d95e45f6e417c95d4d3e6278f4d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.483281 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 (DEFAULT_TABLE table=default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3 [id=56970436dd49486f86fc2fc41f5de5d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.483372 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010d95e45f6e417c95d4d3e6278f4d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.483600 28656 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.483937 28656 tablet_bootstrap.cc:654] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.484019 27197 catalog_manager.cc:5484] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.484125 27848 raft_consensus.cc:2781] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.484158 27848 raft_consensus.cc:482] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.484174 27848 raft_consensus.cc:3037] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.484385 28655 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.484508 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 09426f1e3db54724a043f24c40f0e9f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.484505 28653 ts_tablet_manager.cc:1875] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.484679 27560 tablet_service.cc:1467] Processing CreateTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.484735 28655 tablet_bootstrap.cc:654] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.484768 27848 raft_consensus.cc:504] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.484769 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02d8134f2684fb5a0ddbb9e685c8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.484890 27848 leader_election.cc:288] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.485237 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58740c221e9b422d9fcff96e2df8e85e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.485289 27449 raft_consensus.cc:3037] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.485618 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e562fa324448f788adaa4f60da3076 with delete_type TABLET_DATA_DELETED (replaced by tablet b02d8134f2684fb5a0ddbb9e685c8f79) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.485895 28653 ts_tablet_manager.cc:1888] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.485903 28661 tablet_replica.cc:330] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.485956 28653 log.cc:1196] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/78e562fa324448f788adaa4f60da3076
I20240503 02:09:16.485976 28661 raft_consensus.cc:2229] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.486007 28661 raft_consensus.cc:2258] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.486129 28653 ts_tablet_manager.cc:1909] T 78e562fa324448f788adaa4f60da3076 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.486227 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_def0b15b29374976bcd2300bc83f9c81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.486315 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_def0b15b29374976bcd2300bc83f9c81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.486416 28661 ts_tablet_manager.cc:1875] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.486541 28655 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.486577 28655 ts_tablet_manager.cc:1367] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.486616 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c173d5f4572049b88bbef92d8090e135" } new_table_name: "default.Table_5aa184df608d4c2dafb9752f9a6e615e" modify_external_catalogs: true
I20240503 02:09:16.486677 28740 consensus_queue.cc:1035] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.486699 28655 raft_consensus.cc:348] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.486747 28655 raft_consensus.cc:374] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.486765 28655 raft_consensus.cc:729] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.486799 28655 consensus_queue.cc:260] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.486838 28694 tablet.cc:1722] T 32d965b7253c4f7582e32802e4ec73aa P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.486873 28655 ts_tablet_manager.cc:1398] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.486878 27427 tablet_service.cc:1467] Processing CreateTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.486980 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02d8134f2684fb5a0ddbb9e685c8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.487421 27196 catalog_manager.cc:5484] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.487578 28661 ts_tablet_manager.cc:1888] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:16.487620 28661 log.cc:1196] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/09426f1e3db54724a043f24c40f0e9f1
I20240503 02:09:16.487790 28661 ts_tablet_manager.cc:1909] T 09426f1e3db54724a043f24c40f0e9f1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.485883 27449 raft_consensus.cc:2454] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.487987 28655 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.488233 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58740c221e9b422d9fcff96e2df8e85e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.488274 27524 leader_election.cc:302] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.488289 27723 raft_consensus.cc:3037] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.488318 28655 tablet_bootstrap.cc:654] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.488438 29167 tablet.cc:1722] T 32d965b7253c4f7582e32802e4ec73aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.488490 28410 raft_consensus.cc:2781] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.488534 28410 raft_consensus.cc:686] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.488567 28410 consensus_queue.cc:237] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.488713 28664 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.488749 28664 ts_tablet_manager.cc:1367] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240503 02:09:16.488854 28664 raft_consensus.cc:348] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.488916 28664 raft_consensus.cc:374] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.488934 28664 raft_consensus.cc:729] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.488958 28664 consensus_queue.cc:260] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.489037 28664 ts_tablet_manager.cc:1398] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.489079 28664 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.489159 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 78e562fa324448f788adaa4f60da3076 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:16.489485 28664 tablet_bootstrap.cc:654] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.490906 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_56cf95f3b37f46e7b3ff04cadd5c75e0" } new_table_name: "default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e" modify_external_catalogs: true
I20240503 02:09:16.488845 27723 raft_consensus.cc:2454] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.491358 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_d27da7c2acc24478a93b726989a2d2b9 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
I20240503 02:09:16.491478 28653 tablet_replica.cc:330] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.491528 28653 raft_consensus.cc:2229] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.491557 28653 raft_consensus.cc:2258] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.491736 28710 tablet.cc:1722] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.491778 28654 tablet_replica.cc:330] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.491827 28654 raft_consensus.cc:2229] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.491878 28654 raft_consensus.cc:2258] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.491964 27692 tablet_service.cc:1467] Processing CreateTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (DEFAULT_TABLE table=default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.492034 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02d8134f2684fb5a0ddbb9e685c8f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.492090 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 78e562fa324448f788adaa4f60da3076 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:16.492551 28694 tablet.cc:1722] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.492831 28710 tablet.cc:1722] T 32d965b7253c4f7582e32802e4ec73aa P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.493468 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 34c4d03198554e4aa7a11125f851debc (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.493625 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 12d29bd82e234761a76b1fc59cedbe1c (DEFAULT_TABLE table=default.Table_def0b15b29374976bcd2300bc83f9c81 [id=af530775a302468796ce4cde6e07f9c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.493708 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d29bd82e234761a76b1fc59cedbe1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.494714 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 12d29bd82e234761a76b1fc59cedbe1c (DEFAULT_TABLE table=default.Table_def0b15b29374976bcd2300bc83f9c81 [id=af530775a302468796ce4cde6e07f9c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.494819 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d29bd82e234761a76b1fc59cedbe1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.494967 27198 catalog_manager.cc:5484] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.495079 28653 ts_tablet_manager.cc:1875] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.495739 28654 ts_tablet_manager.cc:1875] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.496063 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 12d29bd82e234761a76b1fc59cedbe1c (DEFAULT_TABLE table=default.Table_def0b15b29374976bcd2300bc83f9c81 [id=af530775a302468796ce4cde6e07f9c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.496152 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d29bd82e234761a76b1fc59cedbe1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.496227 28653 ts_tablet_manager.cc:1888] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:16.496273 28653 log.cc:1196] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/09426f1e3db54724a043f24c40f0e9f1
I20240503 02:09:16.496383 29167 tablet.cc:1722] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.496434 28653 ts_tablet_manager.cc:1909] T 09426f1e3db54724a043f24c40f0e9f1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.496898 28654 ts_tablet_manager.cc:1888] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.496948 28654 log.cc:1196] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/78e562fa324448f788adaa4f60da3076
I20240503 02:09:16.497117 28654 ts_tablet_manager.cc:1909] T 78e562fa324448f788adaa4f60da3076 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.497570 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet b280b918700342319e9484cdb1c4b949 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.497691 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 78e562fa324448f788adaa4f60da3076 (table default.Table_4d99d803f2ac4b599aea7043cceb1506 [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:16.497762 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 09426f1e3db54724a043f24c40f0e9f1 (table default.Table_d27da7c2acc24478a93b726989a2d2b9 [id=7a32c89ef97847cca4b48e7891d95a89]) successfully deleted
I20240503 02:09:16.497996 28694 tablet.cc:1722] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.498019 28656 tablet_bootstrap.cc:492] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.498055 28656 ts_tablet_manager.cc:1367] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.000s
I20240503 02:09:16.498173 28656 raft_consensus.cc:348] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.498219 28656 raft_consensus.cc:374] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.498235 28656 raft_consensus.cc:729] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.498270 28656 consensus_queue.cc:260] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.498381 28656 ts_tablet_manager.cc:1398] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.498422 28655 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.498440 28656 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.498461 28655 ts_tablet_manager.cc:1367] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:16.498579 28655 raft_consensus.cc:348] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.498631 28655 raft_consensus.cc:374] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.498656 28655 raft_consensus.cc:729] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.498683 28655 consensus_queue.cc:260] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.498776 28656 tablet_bootstrap.cc:654] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.498782 28655 ts_tablet_manager.cc:1398] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.498848 28655 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.499188 28655 tablet_bootstrap.cc:654] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.499212 28656 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.499249 28656 ts_tablet_manager.cc:1367] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.499359 28656 raft_consensus.cc:348] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.499421 28656 raft_consensus.cc:374] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.499441 28656 raft_consensus.cc:729] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.499473 28656 consensus_queue.cc:260] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.499593 28656 ts_tablet_manager.cc:1398] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.499622 28664 tablet_bootstrap.cc:492] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.499656 28664 ts_tablet_manager.cc:1367] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:16.499660 28656 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.499789 28664 raft_consensus.cc:348] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.499840 28664 raft_consensus.cc:374] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.499855 28664 raft_consensus.cc:729] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.499841 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_37701508cc6c4db8b07876a048312845" } new_table_name: "default.Table_6bb6cbabe9a24968a5d1a05075cd5689" modify_external_catalogs: true
I20240503 02:09:16.499903 28664 consensus_queue.cc:260] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.499958 28656 tablet_bootstrap.cc:654] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.499984 28664 ts_tablet_manager.cc:1398] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.500046 28664 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.500257 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_6dc41f3a71694c1e854ddc70851cd997 [id=a8cd12a3c7364147b9ce763639338b54] alter complete (version 1)
I20240503 02:09:16.500358 28664 tablet_bootstrap.cc:654] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.500547 28409 raft_consensus.cc:482] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.500588 28409 raft_consensus.cc:504] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.500679 28409 leader_election.cc:288] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.500892 28710 tablet.cc:1722] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.501048 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6] alter complete (version 1)
I20240503 02:09:16.501077 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b832fd89187a4f9c8e30d646c0541986" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.501137 27723 raft_consensus.cc:2454] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.501154 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b280b918700342319e9484cdb1c4b949
I20240503 02:09:16.501199 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b832fd89187a4f9c8e30d646c0541986" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.501230 27201 catalog_manager.cc:6554] ReplaceTablet: tablet b280b918700342319e9484cdb1c4b949 deleted and replaced by tablet 19e5d244a9364cd595a1089c421d9a06
I20240503 02:09:16.501245 27587 raft_consensus.cc:2454] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.501569 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet b280b918700342319e9484cdb1c4b949 with delete_type TABLET_DATA_DELETED (replaced by tablet 19e5d244a9364cd595a1089c421d9a06) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.501634 28661 tablet_replica.cc:330] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.501680 28661 raft_consensus.cc:2229] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.501718 28661 raft_consensus.cc:2258] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.501770 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet b280b918700342319e9484cdb1c4b949 with delete_type TABLET_DATA_DELETED (replaced by tablet 19e5d244a9364cd595a1089c421d9a06) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.501838 28654 tablet_replica.cc:330] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.501892 28654 raft_consensus.cc:2229] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.501919 28654 raft_consensus.cc:2258] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.501909 27388 leader_election.cc:302] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.502269 28409 raft_consensus.cc:2781] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.502305 28409 raft_consensus.cc:482] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.502321 28409 raft_consensus.cc:3037] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.502418 29167 tablet.cc:1722] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.501636 28693 consensus_queue.cc:1035] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:09:16.502857 28661 ts_tablet_manager.cc:1875] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.502923 28409 raft_consensus.cc:504] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.503019 28409 leader_election.cc:288] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.503089 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet b280b918700342319e9484cdb1c4b949 with delete_type TABLET_DATA_DELETED (replaced by tablet 19e5d244a9364cd595a1089c421d9a06) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.503163 28653 tablet_replica.cc:330] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.503202 28653 raft_consensus.cc:2229] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.503229 28653 raft_consensus.cc:2258] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.503572 28654 ts_tablet_manager.cc:1875] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.504042 28661 ts_tablet_manager.cc:1888] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.504084 28661 log.cc:1196] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b280b918700342319e9484cdb1c4b949
I20240503 02:09:16.504261 28661 ts_tablet_manager.cc:1909] T b280b918700342319e9484cdb1c4b949 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.504706 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b832fd89187a4f9c8e30d646c0541986" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.504760 27723 raft_consensus.cc:3037] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.505367 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b280b918700342319e9484cdb1c4b949 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]) successfully deleted
I20240503 02:09:16.505400 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b832fd89187a4f9c8e30d646c0541986" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.505450 27587 raft_consensus.cc:3037] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.505779 28654 ts_tablet_manager.cc:1888] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.505831 28654 log.cc:1196] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b280b918700342319e9484cdb1c4b949
I20240503 02:09:16.505975 27587 raft_consensus.cc:2454] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.506136 27386 leader_election.cc:302] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.506203 28685 raft_consensus.cc:2781] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.506233 28685 raft_consensus.cc:686] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.506275 27723 raft_consensus.cc:2454] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.506266 28685 consensus_queue.cc:237] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.506384 28655 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.506418 28655 ts_tablet_manager.cc:1367] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.506526 28655 raft_consensus.cc:348] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.506572 28655 raft_consensus.cc:374] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.506589 28655 raft_consensus.cc:729] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.506624 28655 consensus_queue.cc:260] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.506700 28655 ts_tablet_manager.cc:1398] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.505983 28654 ts_tablet_manager.cc:1909] T b280b918700342319e9484cdb1c4b949 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.507429 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b280b918700342319e9484cdb1c4b949 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]) successfully deleted
I20240503 02:09:16.507777 28664 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.507778 28656 tablet_bootstrap.cc:492] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.507828 28656 ts_tablet_manager.cc:1367] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.507830 28664 ts_tablet_manager.cc:1367] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:16.507930 28656 raft_consensus.cc:348] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.507980 28656 raft_consensus.cc:374] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.507956 28664 raft_consensus.cc:348] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.507998 28656 raft_consensus.cc:729] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.508003 28664 raft_consensus.cc:374] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.508023 28664 raft_consensus.cc:729] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.508033 28656 consensus_queue.cc:260] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.508055 28664 consensus_queue.cc:260] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.508105 28656 ts_tablet_manager.cc:1398] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.508129 28664 ts_tablet_manager.cc:1398] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.508153 28656 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.508160 28653 ts_tablet_manager.cc:1875] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.508183 28664 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.508391 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5fdda3bf91cc41fd941eee821d4ec456" } modify_external_catalogs: true
I20240503 02:09:16.508445 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5fdda3bf91cc41fd941eee821d4ec456" } modify_external_catalogs: true
I20240503 02:09:16.508472 28656 tablet_bootstrap.cc:654] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.508617 28664 tablet_bootstrap.cc:654] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.508953 28656 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.508987 28656 ts_tablet_manager.cc:1367] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.509569 28656 raft_consensus.cc:348] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.509737 28656 raft_consensus.cc:374] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.509812 28656 raft_consensus.cc:729] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.509994 28656 consensus_queue.cc:260] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.510159 29167 tablet.cc:1722] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.510264 28653 ts_tablet_manager.cc:1888] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.510275 28656 ts_tablet_manager.cc:1398] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.510263 28694 tablet.cc:1722] T 9dc07eb64969465fb2b152337dc82851 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.510310 28653 log.cc:1196] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b280b918700342319e9484cdb1c4b949
I20240503 02:09:16.510505 28653 ts_tablet_manager.cc:1909] T b280b918700342319e9484cdb1c4b949 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.510761 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b280b918700342319e9484cdb1c4b949 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]) successfully deleted
I20240503 02:09:16.511231 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c08af667e8174205a3e31c4892085fb9
I20240503 02:09:16.511399 28664 tablet_bootstrap.cc:492] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.511411 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet c08af667e8174205a3e31c4892085fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.511435 28664 ts_tablet_manager.cc:1367] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.511471 28653 tablet_replica.cc:330] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.511518 28653 raft_consensus.cc:2229] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.511564 28653 raft_consensus.cc:2258] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.511582 28664 raft_consensus.cc:348] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.511631 28664 raft_consensus.cc:374] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.511646 28664 raft_consensus.cc:729] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.511651 28653 ts_tablet_manager.cc:1875] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.511693 28664 consensus_queue.cc:260] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.511778 28664 ts_tablet_manager.cc:1398] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.512715 28653 ts_tablet_manager.cc:1888] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.512789 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 19e5d244a9364cd595a1089c421d9a06 (DEFAULT_TABLE table=default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.512850 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19e5d244a9364cd595a1089c421d9a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.513131 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 19e5d244a9364cd595a1089c421d9a06 (DEFAULT_TABLE table=default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.513213 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19e5d244a9364cd595a1089c421d9a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.513844 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet c08af667e8174205a3e31c4892085fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.513857 28655 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.513927 28654 tablet_replica.cc:330] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.513962 28653 log.cc:1196] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c08af667e8174205a3e31c4892085fb9
I20240503 02:09:16.513979 28654 raft_consensus.cc:2229] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.514011 28654 raft_consensus.cc:2258] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.514109 28654 ts_tablet_manager.cc:1875] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.514181 28655 tablet_bootstrap.cc:654] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.514251 27848 raft_consensus.cc:482] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.514286 27848 raft_consensus.cc:504] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.514362 27848 leader_election.cc:288] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.514196 28653 ts_tablet_manager.cc:1909] T c08af667e8174205a3e31c4892085fb9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.515245 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.515314 27449 raft_consensus.cc:2379] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:16.515344 28710 tablet.cc:1722] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.515576 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a166dbdc3574a0183e3cc4896307be2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.515609 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]) successfully deleted
I20240503 02:09:16.515930 28656 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.516486 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 203665395f3745f196fb393dd82b8fc9 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.516611 28654 ts_tablet_manager.cc:1888] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.516641 28656 tablet_bootstrap.cc:654] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.516656 28654 log.cc:1196] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c08af667e8174205a3e31c4892085fb9
I20240503 02:09:16.516709 27524 leader_election.cc:302] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:16.516795 28507 raft_consensus.cc:3037] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.516849 28654 ts_tablet_manager.cc:1909] T c08af667e8174205a3e31c4892085fb9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.516943 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet b280b918700342319e9484cdb1c4b949 with delete_type TABLET_DATA_DELETED (replaced by tablet 19e5d244a9364cd595a1089c421d9a06) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.517372 28507 raft_consensus.cc:2726] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:16.517436 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet c08af667e8174205a3e31c4892085fb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.517470 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 19e5d244a9364cd595a1089c421d9a06 (DEFAULT_TABLE table=default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.517526 28766 raft_consensus.cc:482] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.517546 27853 raft_consensus.cc:482] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.517567 28766 raft_consensus.cc:504] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.517587 27853 raft_consensus.cc:504] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.517653 28766 leader_election.cc:288] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.517678 27853 leader_election.cc:288] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.517556 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19e5d244a9364cd595a1089c421d9a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.517963 28766 consensus_queue.cc:1035] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.518198 28661 tablet_replica.cc:330] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.518263 28661 raft_consensus.cc:2229] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.518299 28661 raft_consensus.cc:2258] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.518409 28661 ts_tablet_manager.cc:1875] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.518429 27201 catalog_manager.cc:5484] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.518640 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_6bb6cbabe9a24968a5d1a05075cd5689 [id=dcaa66ed9577423ebe462a9302aa4a91] alter complete (version 1)
I20240503 02:09:16.518771 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b402f1be4d4fab80a0cef3392725bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.518827 27587 raft_consensus.cc:2454] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.518904 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b402f1be4d4fab80a0cef3392725bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.518951 27723 raft_consensus.cc:2454] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.519101 27388 leader_election.cc:302] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.519135 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a710cef4241b47d788dced187487c7cd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.519160 28766 raft_consensus.cc:2781] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.519189 28766 raft_consensus.cc:482] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.519188 27587 raft_consensus.cc:2454] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.519209 28766 raft_consensus.cc:3037] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.519598 28661 ts_tablet_manager.cc:1888] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.519639 28661 log.cc:1196] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c08af667e8174205a3e31c4892085fb9
I20240503 02:09:16.519651 28664 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.519837 28661 ts_tablet_manager.cc:1909] T c08af667e8174205a3e31c4892085fb9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.519841 28766 raft_consensus.cc:504] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.519948 28766 leader_election.cc:288] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.519989 28664 tablet_bootstrap.cc:654] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.520167 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_61d9e998f8ef48daaec7c5087f31e260" } modify_external_catalogs: true
I20240503 02:09:16.520223 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_61d9e998f8ef48daaec7c5087f31e260" } modify_external_catalogs: true
I20240503 02:09:16.520263 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b402f1be4d4fab80a0cef3392725bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.520313 27723 raft_consensus.cc:3037] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.520817 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a710cef4241b47d788dced187487c7cd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.520889 27449 raft_consensus.cc:2454] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.520892 27723 raft_consensus.cc:2454] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
W20240503 02:09:16.520961 27189 catalog_manager.cc:4797] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet b280b918700342319e9484cdb1c4b949 because the tablet was not found. No further retry: Not found: Tablet not found: b280b918700342319e9484cdb1c4b949
I20240503 02:09:16.521041 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]) successfully deleted
I20240503 02:09:16.521126 27388 leader_election.cc:302] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.521144 28664 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.521184 28685 raft_consensus.cc:2781] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.521200 28664 ts_tablet_manager.cc:1367] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.521219 28685 raft_consensus.cc:686] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.521253 28685 consensus_queue.cc:237] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.521415 28664 raft_consensus.cc:348] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.521473 28664 raft_consensus.cc:374] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.521492 28664 raft_consensus.cc:729] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.521524 28664 consensus_queue.cc:260] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.521605 28664 ts_tablet_manager.cc:1398] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.521715 27659 leader_election.cc:302] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.521936 27853 raft_consensus.cc:2781] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.521975 27853 raft_consensus.cc:482] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.521992 27853 raft_consensus.cc:3037] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.522276 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 694e3a4f847e4f2d9d386cbb5c09842c
I20240503 02:09:16.522492 27853 raft_consensus.cc:504] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.522555 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.522589 27853 leader_election.cc:288] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.522747 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.522774 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e3a4f847e4f2d9d386cbb5c09842c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.522835 28661 tablet_replica.cc:330] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.522838 28653 tablet_replica.cc:330] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.522881 28653 raft_consensus.cc:2229] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.522884 28661 raft_consensus.cc:2229] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.522913 28661 raft_consensus.cc:2258] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.522910 28653 raft_consensus.cc:2258] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.523028 28653 ts_tablet_manager.cc:1875] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.523133 28661 ts_tablet_manager.cc:1875] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.523133 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a710cef4241b47d788dced187487c7cd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.523185 27449 raft_consensus.cc:3037] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.523696 27449 raft_consensus.cc:2454] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.523733 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
W20240503 02:09:16.523802 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 19e5d244a9364cd595a1089c421d9a06 consensus state has no leader
I20240503 02:09:16.523828 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b402f1be4d4fab80a0cef3392725bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.523878 27583 raft_consensus.cc:3037] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.524516 27583 raft_consensus.cc:2454] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.524520 28653 ts_tablet_manager.cc:1888] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.524559 28653 log.cc:1196] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/694e3a4f847e4f2d9d386cbb5c09842c
I20240503 02:09:16.524593 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a710cef4241b47d788dced187487c7cd" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.524542 27201 catalog_manager.cc:5484] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.524636 27587 raft_consensus.cc:3037] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.524706 28661 ts_tablet_manager.cc:1888] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.524749 28661 log.cc:1196] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/694e3a4f847e4f2d9d386cbb5c09842c
I20240503 02:09:16.524772 28653 ts_tablet_manager.cc:1909] T 694e3a4f847e4f2d9d386cbb5c09842c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.525004 28661 ts_tablet_manager.cc:1909] T 694e3a4f847e4f2d9d386cbb5c09842c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.525230 27587 raft_consensus.cc:2454] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.525238 28655 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.525276 28655 ts_tablet_manager.cc:1367] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.000s
I20240503 02:09:16.525442 28655 raft_consensus.cc:348] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.525497 28655 raft_consensus.cc:374] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.525513 28655 raft_consensus.cc:729] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.525547 28655 consensus_queue.cc:260] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.525621 28655 ts_tablet_manager.cc:1398] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.525758 27656 leader_election.cc:302] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.525846 27853 raft_consensus.cc:2781] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.525884 27853 raft_consensus.cc:686] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.525918 27853 consensus_queue.cc:237] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.526005 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 694e3a4f847e4f2d9d386cbb5c09842c (table default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:16.526407 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 694e3a4f847e4f2d9d386cbb5c09842c (table default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:16.526724 28654 tablet_replica.cc:330] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.526784 28654 raft_consensus.cc:2229] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.526830 28654 raft_consensus.cc:2258] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.526929 28654 ts_tablet_manager.cc:1875] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.527055 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c08af667e8174205a3e31c4892085fb9 (table default.Table_5fdda3bf91cc41fd941eee821d4ec456 [id=784bc279fe5c41f68dc2bd9ecbbb887d]) successfully deleted
I20240503 02:09:16.527104 27201 catalog_manager.cc:5484] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.527895 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
I20240503 02:09:16.528467 28654 ts_tablet_manager.cc:1888] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.528517 28654 log.cc:1196] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/694e3a4f847e4f2d9d386cbb5c09842c
I20240503 02:09:16.528700 28654 ts_tablet_manager.cc:1909] T 694e3a4f847e4f2d9d386cbb5c09842c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.528739 28656 tablet_bootstrap.cc:492] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.528776 28656 ts_tablet_manager.cc:1367] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:16.528875 28409 consensus_queue.cc:1035] T a8ce961ecae44ca18642f64cf3f02977 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.528895 28656 raft_consensus.cc:348] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.528945 28656 raft_consensus.cc:374] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.528959 28656 raft_consensus.cc:729] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.528991 28656 consensus_queue.cc:260] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.529070 28656 ts_tablet_manager.cc:1398] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.529690 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 8f852b718f524467947b42f653f3530b from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.530114 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 694e3a4f847e4f2d9d386cbb5c09842c (table default.Table_61d9e998f8ef48daaec7c5087f31e260 [id=1d806aada5814e4d86568e5488045240]) successfully deleted
I20240503 02:09:16.530318 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
I20240503 02:09:16.530946 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 3c6ff433791d4ab0a00933702258a55a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.531164 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8f852b718f524467947b42f653f3530b
I20240503 02:09:16.531308 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 8f852b718f524467947b42f653f3530b deleted and replaced by tablet 1f6d82db9bc94f95ad0e602a43bcdc87
I20240503 02:09:16.531464 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f852b718f524467947b42f653f3530b with delete_type TABLET_DATA_DELETED (replaced by tablet 1f6d82db9bc94f95ad0e602a43bcdc87) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.531533 28653 tablet_replica.cc:330] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.531596 28653 raft_consensus.cc:2229] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.531634 28653 pending_rounds.cc:62] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:16.531649 28653 pending_rounds.cc:68] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420032641515520 op_type: NO_OP noop_request { }
I20240503 02:09:16.531651 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f852b718f524467947b42f653f3530b with delete_type TABLET_DATA_DELETED (replaced by tablet 1f6d82db9bc94f95ad0e602a43bcdc87) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.531715 28661 tablet_replica.cc:330] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.531680 28653 raft_consensus.cc:2866] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.531780 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f852b718f524467947b42f653f3530b with delete_type TABLET_DATA_DELETED (replaced by tablet 1f6d82db9bc94f95ad0e602a43bcdc87) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.531819 28653 raft_consensus.cc:2258] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.531893 28654 tablet_replica.cc:330] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.531841 28661 raft_consensus.cc:2229] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.531953 28654 raft_consensus.cc:2229] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.531982 28654 raft_consensus.cc:2258] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.531982 28661 raft_consensus.cc:2258] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.532157 28661 ts_tablet_manager.cc:1875] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.532220 28654 ts_tablet_manager.cc:1875] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.532966 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6d82db9bc94f95ad0e602a43bcdc87 (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.533028 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3c6ff433791d4ab0a00933702258a55a
I20240503 02:09:16.533056 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6d82db9bc94f95ad0e602a43bcdc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.533032 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6d82db9bc94f95ad0e602a43bcdc87 (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.533094 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 3c6ff433791d4ab0a00933702258a55a deleted and replaced by tablet f471bbd597ef4763bf54eac9fb7ccc9a
I20240503 02:09:16.533128 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6d82db9bc94f95ad0e602a43bcdc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.533500 28654 ts_tablet_manager.cc:1888] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.533545 28654 log.cc:1196] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8f852b718f524467947b42f653f3530b
I20240503 02:09:16.533697 28654 ts_tablet_manager.cc:1909] T 8f852b718f524467947b42f653f3530b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.533802 28653 ts_tablet_manager.cc:1875] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.533933 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6ff433791d4ab0a00933702258a55a with delete_type TABLET_DATA_DELETED (replaced by tablet f471bbd597ef4763bf54eac9fb7ccc9a) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.534081 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8f852b718f524467947b42f653f3530b (table default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]) successfully deleted
I20240503 02:09:16.534202 28656 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.534324 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6ff433791d4ab0a00933702258a55a with delete_type TABLET_DATA_DELETED (replaced by tablet f471bbd597ef4763bf54eac9fb7ccc9a) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.534413 28655 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.534418 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1f6d82db9bc94f95ad0e602a43bcdc87 (DEFAULT_TABLE table=default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.534533 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f6d82db9bc94f95ad0e602a43bcdc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.534560 28656 tablet_bootstrap.cc:654] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.534744 28655 tablet_bootstrap.cc:654] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.535009 28653 ts_tablet_manager.cc:1888] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.535059 28653 log.cc:1196] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8f852b718f524467947b42f653f3530b
I20240503 02:09:16.535243 28653 ts_tablet_manager.cc:1909] T 8f852b718f524467947b42f653f3530b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.535351 28656 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.535389 28656 ts_tablet_manager.cc:1367] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.535527 28656 raft_consensus.cc:348] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.535581 28656 raft_consensus.cc:374] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.535599 28656 raft_consensus.cc:729] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.535617 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8f852b718f524467947b42f653f3530b (table default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]) successfully deleted
I20240503 02:09:16.535636 28656 consensus_queue.cc:260] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.535673 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6ff433791d4ab0a00933702258a55a with delete_type TABLET_DATA_DELETED (replaced by tablet f471bbd597ef4763bf54eac9fb7ccc9a) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.535727 28656 ts_tablet_manager.cc:1398] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.535737 28654 tablet_replica.cc:330] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.535781 28654 raft_consensus.cc:2229] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.535809 28654 raft_consensus.cc:2258] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.535864 28653 tablet_replica.cc:330] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.535873 28664 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.535969 28653 raft_consensus.cc:2229] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.536020 28653 pending_rounds.cc:62] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:16.536037 28653 pending_rounds.cc:68] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420032919281664 op_type: NO_OP noop_request { }
I20240503 02:09:16.536072 28653 raft_consensus.cc:2866] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.536090 28653 raft_consensus.cc:2258] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.536247 28664 tablet_bootstrap.cc:654] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.536340 28655 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.536375 28655 ts_tablet_manager.cc:1367] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.536505 28655 raft_consensus.cc:348] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.536554 28655 raft_consensus.cc:374] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.536572 28655 raft_consensus.cc:729] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.536602 28655 consensus_queue.cc:260] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.536602 28654 ts_tablet_manager.cc:1875] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.536713 28655 ts_tablet_manager.cc:1398] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.536813 27560 tablet_service.cc:1467] Processing CreateTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.536895 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f471bbd597ef4763bf54eac9fb7ccc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.537444 27427 tablet_service.cc:1467] Processing CreateTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.537536 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f471bbd597ef4763bf54eac9fb7ccc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.537776 28654 ts_tablet_manager.cc:1888] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.537824 28654 log.cc:1196] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3c6ff433791d4ab0a00933702258a55a
I20240503 02:09:16.537967 28654 ts_tablet_manager.cc:1909] T 3c6ff433791d4ab0a00933702258a55a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.538070 28664 tablet_bootstrap.cc:492] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.538110 28664 ts_tablet_manager.cc:1367] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.538156 27703 tablet_service.cc:1467] Processing CreateTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a (DEFAULT_TABLE table=default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.538240 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f471bbd597ef4763bf54eac9fb7ccc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.538254 28664 raft_consensus.cc:348] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.538309 28664 raft_consensus.cc:374] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.538327 28664 raft_consensus.cc:729] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.538368 28664 consensus_queue.cc:260] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.538475 28664 ts_tablet_manager.cc:1398] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.539630 28655 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.539707 28664 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.539968 28661 ts_tablet_manager.cc:1888] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.540011 28661 log.cc:1196] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8f852b718f524467947b42f653f3530b
I20240503 02:09:16.540035 28655 tablet_bootstrap.cc:654] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.540067 28664 tablet_bootstrap.cc:654] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.540153 28661 ts_tablet_manager.cc:1909] T 8f852b718f524467947b42f653f3530b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.540547 28655 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.540589 28655 ts_tablet_manager.cc:1367] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.540655 28653 ts_tablet_manager.cc:1875] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.540707 28655 raft_consensus.cc:348] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.540758 28655 raft_consensus.cc:374] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.540776 28655 raft_consensus.cc:729] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.540822 28655 consensus_queue.cc:260] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.540910 28655 ts_tablet_manager.cc:1398] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.541271 28664 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.541304 28664 ts_tablet_manager.cc:1367] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.541369 28656 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.541448 28664 raft_consensus.cc:348] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.541505 28664 raft_consensus.cc:374] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.541522 28664 raft_consensus.cc:729] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.541561 28664 consensus_queue.cc:260] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.541652 28664 ts_tablet_manager.cc:1398] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.541718 28656 tablet_bootstrap.cc:654] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.542292 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3c6ff433791d4ab0a00933702258a55a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.542702 28653 ts_tablet_manager.cc:1888] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.542757 28653 log.cc:1196] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3c6ff433791d4ab0a00933702258a55a
I20240503 02:09:16.542754 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8f852b718f524467947b42f653f3530b (table default.Table_0294e95bfb874319bf21ee6eb1b62d63 [id=0651c839d47a46edade1d77c27500e81]) successfully deleted
I20240503 02:09:16.542737 28661 tablet_replica.cc:330] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.542948 28653 ts_tablet_manager.cc:1909] T 3c6ff433791d4ab0a00933702258a55a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.542968 28661 raft_consensus.cc:2229] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.543098 28661 raft_consensus.cc:2258] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.543370 27587 raft_consensus.cc:1261] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.543458 28661 ts_tablet_manager.cc:1875] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.543454 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3c6ff433791d4ab0a00933702258a55a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.543525 27449 raft_consensus.cc:1261] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.543534 27853 consensus_queue.cc:1035] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.543722 27853 consensus_queue.cc:1035] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.544507 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:16.544904 28661 ts_tablet_manager.cc:1888] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.545018 28661 log.cc:1196] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3c6ff433791d4ab0a00933702258a55a
I20240503 02:09:16.545097 29181 mvcc.cc:205] Tried to move back new op lower bound from 7023420033199915008 to 7023420032054235136. Current Snapshot: MvccSnapshot[applied={T|T < 7023420033199915008}]
I20240503 02:09:16.545372 28694 tablet.cc:1722] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.545617 28661 ts_tablet_manager.cc:1909] T 3c6ff433791d4ab0a00933702258a55a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.545740 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:16.545917 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3c6ff433791d4ab0a00933702258a55a (table default.Table_b4b67bcbc8de4b17a042600c659d7c34 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:16.546126 28656 tablet_bootstrap.cc:492] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.546172 28656 ts_tablet_manager.cc:1367] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.546239 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]). Expected version 1 got 0
I20240503 02:09:16.546298 28656 raft_consensus.cc:348] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.546353 28656 raft_consensus.cc:374] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.546370 28656 raft_consensus.cc:729] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.546407 28656 consensus_queue.cc:260] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.546499 28656 ts_tablet_manager.cc:1398] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.546900 28710 tablet.cc:1722] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.547370 29167 tablet.cc:1722] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.552963 28693 raft_consensus.cc:482] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.553018 28693 raft_consensus.cc:504] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.553133 28693 leader_election.cc:288] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.553228 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1177866c9e94ce9bfdc45ecbc6172d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.553299 27449 raft_consensus.cc:2454] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.553282 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1177866c9e94ce9bfdc45ecbc6172d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.553390 27587 raft_consensus.cc:2454] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.553433 27659 leader_election.cc:302] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.553524 28693 raft_consensus.cc:2781] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.553550 28693 raft_consensus.cc:482] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.553570 28693 raft_consensus.cc:3037] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.554000 28693 raft_consensus.cc:504] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.554116 28693 leader_election.cc:288] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.554234 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1177866c9e94ce9bfdc45ecbc6172d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.554297 27587 raft_consensus.cc:3037] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.554306 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1177866c9e94ce9bfdc45ecbc6172d2" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.554373 27449 raft_consensus.cc:3037] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.554610 28507 raft_consensus.cc:482] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.554659 28507 raft_consensus.cc:504] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.554761 28507 leader_election.cc:288] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.554857 27587 raft_consensus.cc:2454] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.554874 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9654b6fa66a84fffb440477a95a0f3a9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.554924 27453 raft_consensus.cc:2454] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.554991 27656 leader_election.cc:302] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.555022 27449 raft_consensus.cc:2454] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.555027 27524 leader_election.cc:302] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.555058 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9654b6fa66a84fffb440477a95a0f3a9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.555104 27723 raft_consensus.cc:2454] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.555112 28507 raft_consensus.cc:2781] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.555140 28507 raft_consensus.cc:482] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.555148 28693 raft_consensus.cc:2781] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.555157 28507 raft_consensus.cc:3037] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.555178 28693 raft_consensus.cc:686] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.555212 28693 consensus_queue.cc:237] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.555744 28507 raft_consensus.cc:504] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.555855 28507 leader_election.cc:288] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.555928 28409 raft_consensus.cc:482] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.555959 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9654b6fa66a84fffb440477a95a0f3a9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.555972 28409 raft_consensus.cc:504] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.556030 27449 raft_consensus.cc:3037] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.556114 28409 leader_election.cc:288] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.556097 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9654b6fa66a84fffb440477a95a0f3a9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.556161 27723 raft_consensus.cc:3037] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.556213 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d29bd82e234761a76b1fc59cedbe1c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.556265 27719 raft_consensus.cc:2454] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.556298 27196 catalog_manager.cc:5484] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.556371 27388 leader_election.cc:302] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.556429 28409 raft_consensus.cc:2781] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.556449 28409 raft_consensus.cc:482] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.556468 28409 raft_consensus.cc:3037] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.556593 27449 raft_consensus.cc:2454] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.556707 27723 raft_consensus.cc:2454] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.556834 27524 leader_election.cc:302] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.556849 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d29bd82e234761a76b1fc59cedbe1c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.556905 27587 raft_consensus.cc:2454] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.557039 28409 raft_consensus.cc:504] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.557150 28409 leader_election.cc:288] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.557581 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d29bd82e234761a76b1fc59cedbe1c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.557636 27723 raft_consensus.cc:3037] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.557713 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d29bd82e234761a76b1fc59cedbe1c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.557750 28507 raft_consensus.cc:2781] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.557770 27587 raft_consensus.cc:3037] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.557794 28507 raft_consensus.cc:686] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.557830 28507 consensus_queue.cc:237] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.558172 27723 raft_consensus.cc:2454] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.558332 27587 raft_consensus.cc:2454] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.558557 27386 leader_election.cc:302] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.558957 28409 raft_consensus.cc:2781] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.559017 28409 raft_consensus.cc:686] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.559060 28409 consensus_queue.cc:237] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.559548 27198 catalog_manager.cc:5484] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.559890 27196 catalog_manager.cc:5484] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
W20240503 02:09:16.559994 27187 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 19e5d244a9364cd595a1089c421d9a06 consensus state has no leader
I20240503 02:09:16.561371 28507 raft_consensus.cc:482] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.561419 28507 raft_consensus.cc:504] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.561511 28507 leader_election.cc:288] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.561643 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ad86edb8f646f5b717463b4fae2263" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.561638 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ad86edb8f646f5b717463b4fae2263" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.561708 27723 raft_consensus.cc:2454] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.561717 27449 raft_consensus.cc:2454] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.561830 27524 leader_election.cc:302] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.561905 28507 raft_consensus.cc:2781] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.561939 28507 raft_consensus.cc:482] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.561970 28507 raft_consensus.cc:3037] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.562409 28507 raft_consensus.cc:504] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.562505 28507 leader_election.cc:288] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.562618 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ad86edb8f646f5b717463b4fae2263" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.562618 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ad86edb8f646f5b717463b4fae2263" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.562696 27723 raft_consensus.cc:3037] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.562696 27449 raft_consensus.cc:3037] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.563210 27723 raft_consensus.cc:2454] T 93ad86edb8f646f5b717463b4fae2263 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.563210 27449 raft_consensus.cc:2454] T 93ad86edb8f646f5b717463b4fae2263 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.563392 27523 leader_election.cc:302] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.563479 28507 raft_consensus.cc:2781] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.563585 28507 raft_consensus.cc:686] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.563630 28507 consensus_queue.cc:237] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.564297 27196 catalog_manager.cc:5484] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.567101 28767 raft_consensus.cc:482] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.567144 28767 raft_consensus.cc:504] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.567245 28767 leader_election.cc:288] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.567378 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32edb5ccf44199fce2ec6a2c8cf63" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.567412 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32edb5ccf44199fce2ec6a2c8cf63" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.567437 27587 raft_consensus.cc:2454] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.567461 27723 raft_consensus.cc:2454] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.567553 27386 leader_election.cc:302] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.567637 28767 raft_consensus.cc:2781] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.567677 28767 raft_consensus.cc:482] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.567694 28767 raft_consensus.cc:3037] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.568104 28767 raft_consensus.cc:504] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.568200 28767 leader_election.cc:288] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.568310 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32edb5ccf44199fce2ec6a2c8cf63" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.568365 27587 raft_consensus.cc:3037] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.568347 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e32edb5ccf44199fce2ec6a2c8cf63" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.568390 27723 raft_consensus.cc:3037] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.568773 27587 raft_consensus.cc:2454] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.568812 27723 raft_consensus.cc:2454] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.568924 27386 leader_election.cc:302] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.569007 28767 raft_consensus.cc:2781] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.569057 28767 raft_consensus.cc:686] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.569098 28767 consensus_queue.cc:237] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.569773 27196 catalog_manager.cc:5484] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.569880 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.585362 28507 raft_consensus.cc:482] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.585453 28507 raft_consensus.cc:504] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.585599 28507 leader_election.cc:288] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.585738 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcad4e4abb9a47ac9a6c481e91eb1bf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.585745 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcad4e4abb9a47ac9a6c481e91eb1bf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.585830 27449 raft_consensus.cc:2454] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.585855 27723 raft_consensus.cc:2454] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.585974 27524 leader_election.cc:302] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.586129 28507 raft_consensus.cc:2781] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.586170 28507 raft_consensus.cc:482] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.586199 28507 raft_consensus.cc:3037] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.586794 28507 raft_consensus.cc:504] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.586925 28507 leader_election.cc:288] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.587085 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcad4e4abb9a47ac9a6c481e91eb1bf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.587097 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcad4e4abb9a47ac9a6c481e91eb1bf9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.587173 27723 raft_consensus.cc:3037] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.587196 27449 raft_consensus.cc:3037] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.587788 27449 raft_consensus.cc:2454] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.587826 27723 raft_consensus.cc:2454] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.587956 27524 leader_election.cc:302] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.588071 28507 raft_consensus.cc:2781] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.588125 28507 raft_consensus.cc:686] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.588178 28507 consensus_queue.cc:237] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.588384 28693 raft_consensus.cc:482] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.588449 28693 raft_consensus.cc:504] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.588572 28693 leader_election.cc:288] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.588724 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010d95e45f6e417c95d4d3e6278f4d74" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.588793 27449 raft_consensus.cc:2454] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.588923 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010d95e45f6e417c95d4d3e6278f4d74" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.588992 27587 raft_consensus.cc:2454] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.589111 27659 leader_election.cc:302] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.589241 28693 raft_consensus.cc:2781] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.589277 28693 raft_consensus.cc:482] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.589299 28693 raft_consensus.cc:3037] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.589476 27196 catalog_manager.cc:5484] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.589975 28693 raft_consensus.cc:504] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.590107 28693 leader_election.cc:288] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.590453 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010d95e45f6e417c95d4d3e6278f4d74" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.590524 27587 raft_consensus.cc:3037] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.590529 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010d95e45f6e417c95d4d3e6278f4d74" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.590584 27449 raft_consensus.cc:3037] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.591174 27449 raft_consensus.cc:2454] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.591174 27587 raft_consensus.cc:2454] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.591338 27656 leader_election.cc:302] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.591456 28693 raft_consensus.cc:2781] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.591498 28693 raft_consensus.cc:686] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.591539 28693 consensus_queue.cc:237] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.592628 27196 catalog_manager.cc:5484] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.594753 28774 raft_consensus.cc:482] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.594811 28774 raft_consensus.cc:504] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.594914 28774 leader_election.cc:288] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.595055 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b97787b7d314c39b4ea9e599dbe692e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.595129 27587 raft_consensus.cc:2454] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.595121 28767 raft_consensus.cc:482] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.595124 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b97787b7d314c39b4ea9e599dbe692e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.595193 28767 raft_consensus.cc:504] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.595233 27656 leader_election.cc:302] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.595263 27449 raft_consensus.cc:2454] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.595336 28767 leader_election.cc:288] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.595328 28774 raft_consensus.cc:2781] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.595363 28774 raft_consensus.cc:482] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.595384 28774 raft_consensus.cc:3037] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.595443 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20aa1acdfbf4c27ba2b5a5dbc75483d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.595516 27723 raft_consensus.cc:2454] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.595582 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20aa1acdfbf4c27ba2b5a5dbc75483d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.595633 27587 raft_consensus.cc:2454] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.595626 27388 leader_election.cc:302] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.595717 28767 raft_consensus.cc:2781] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.595778 28767 raft_consensus.cc:482] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.595803 28767 raft_consensus.cc:3037] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.595975 28774 raft_consensus.cc:504] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.596092 28774 leader_election.cc:288] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.596246 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b97787b7d314c39b4ea9e599dbe692e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.596251 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b97787b7d314c39b4ea9e599dbe692e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.596297 27587 raft_consensus.cc:3037] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.596318 27449 raft_consensus.cc:3037] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.596515 28767 raft_consensus.cc:504] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.596634 28767 leader_election.cc:288] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.596875 27587 raft_consensus.cc:2454] T 5b97787b7d314c39b4ea9e599dbe692e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.596904 27449 raft_consensus.cc:2454] T 5b97787b7d314c39b4ea9e599dbe692e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.596957 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20aa1acdfbf4c27ba2b5a5dbc75483d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.597003 27656 leader_election.cc:302] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.597023 27582 raft_consensus.cc:3037] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.597075 28774 raft_consensus.cc:2781] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.597129 28774 raft_consensus.cc:686] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.597179 28774 consensus_queue.cc:237] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.597725 27582 raft_consensus.cc:2454] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.598002 27196 catalog_manager.cc:5484] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.598105 27386 leader_election.cc:302] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.598198 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d20aa1acdfbf4c27ba2b5a5dbc75483d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.598248 27723 raft_consensus.cc:3037] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.598237 28767 raft_consensus.cc:2781] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.598295 28767 raft_consensus.cc:686] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.598338 28767 consensus_queue.cc:237] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.598831 27723 raft_consensus.cc:2454] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.599488 27198 catalog_manager.cc:5484] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.599602 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.605201 28774 raft_consensus.cc:482] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.605253 28774 raft_consensus.cc:504] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.605372 28774 leader_election.cc:288] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.605511 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b7bf524ad049dba8cd61f4cde3cec7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.605540 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b7bf524ad049dba8cd61f4cde3cec7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.605582 27453 raft_consensus.cc:2454] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.605588 27582 raft_consensus.cc:2454] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.605712 27656 leader_election.cc:302] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.605818 28774 raft_consensus.cc:2781] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.605855 28774 raft_consensus.cc:482] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.605872 28774 raft_consensus.cc:3037] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.606305 28774 raft_consensus.cc:504] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.606423 28774 leader_election.cc:288] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.606554 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b7bf524ad049dba8cd61f4cde3cec7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.606577 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44b7bf524ad049dba8cd61f4cde3cec7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.606621 27453 raft_consensus.cc:3037] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.606624 27582 raft_consensus.cc:3037] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.607201 27453 raft_consensus.cc:2454] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.607201 27582 raft_consensus.cc:2454] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.607352 27659 leader_election.cc:302] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.607442 28774 raft_consensus.cc:2781] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.607496 28774 raft_consensus.cc:686] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.607549 28774 consensus_queue.cc:237] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.608428 27198 catalog_manager.cc:5484] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.611192 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_665b0d3aca514edd91cee5dc4f99a3e2" } modify_external_catalogs: true
I20240503 02:09:16.611255 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_665b0d3aca514edd91cee5dc4f99a3e2" } modify_external_catalogs: true
I20240503 02:09:16.612123 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d52224df3392476fa9a4959341cc737a
I20240503 02:09:16.612372 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet d52224df3392476fa9a4959341cc737a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.612473 28661 tablet_replica.cc:330] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.612545 28661 raft_consensus.cc:2229] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.612600 28661 pending_rounds.cc:62] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:16.612617 28661 pending_rounds.cc:68] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420032095088640 op_type: NO_OP noop_request { }
I20240503 02:09:16.612646 28661 raft_consensus.cc:2866] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.612660 28661 raft_consensus.cc:2258] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.612720 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet d52224df3392476fa9a4959341cc737a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.612792 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet d52224df3392476fa9a4959341cc737a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.612864 28654 tablet_replica.cc:330] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.612921 28654 raft_consensus.cc:2229] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.612926 28661 ts_tablet_manager.cc:1875] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.612938 28507 raft_consensus.cc:482] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.612952 28654 raft_consensus.cc:2258] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.612974 28507 raft_consensus.cc:504] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.613013 28653 tablet_replica.cc:330] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.613065 28653 raft_consensus.cc:2229] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.613078 28507 leader_election.cc:288] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.613093 28653 raft_consensus.cc:2258] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.613281 28653 ts_tablet_manager.cc:1875] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.613477 28654 ts_tablet_manager.cc:1875] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.614318 28661 ts_tablet_manager.cc:1888] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.614360 28661 log.cc:1196] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d52224df3392476fa9a4959341cc737a
I20240503 02:09:16.614531 28661 ts_tablet_manager.cc:1909] T d52224df3392476fa9a4959341cc737a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.614612 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f471bbd597ef4763bf54eac9fb7ccc9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.614683 27453 raft_consensus.cc:2454] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.614796 27524 leader_election.cc:302] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.614866 28507 raft_consensus.cc:2781] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.614893 28507 raft_consensus.cc:482] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.614912 28507 raft_consensus.cc:3037] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.614915 28653 ts_tablet_manager.cc:1888] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.614913 28654 ts_tablet_manager.cc:1888] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.614962 28653 log.cc:1196] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d52224df3392476fa9a4959341cc737a
I20240503 02:09:16.614991 28654 log.cc:1196] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d52224df3392476fa9a4959341cc737a
I20240503 02:09:16.615119 28653 ts_tablet_manager.cc:1909] T d52224df3392476fa9a4959341cc737a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.615146 28654 ts_tablet_manager.cc:1909] T d52224df3392476fa9a4959341cc737a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.615259 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f471bbd597ef4763bf54eac9fb7ccc9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.615316 27723 raft_consensus.cc:2454] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.615525 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d52224df3392476fa9a4959341cc737a (table default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]) successfully deleted
I20240503 02:09:16.615595 28507 raft_consensus.cc:504] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.615777 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d52224df3392476fa9a4959341cc737a (table default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]) successfully deleted
I20240503 02:09:16.615782 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d52224df3392476fa9a4959341cc737a (table default.Table_665b0d3aca514edd91cee5dc4f99a3e2 [id=dc1d156a36a8497eab6744d82007289b]) successfully deleted
I20240503 02:09:16.615800 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f471bbd597ef4763bf54eac9fb7ccc9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.615850 27453 raft_consensus.cc:3037] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.615870 28507 leader_election.cc:288] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.616283 27453 raft_consensus.cc:2454] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.616364 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f471bbd597ef4763bf54eac9fb7ccc9a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.616420 27723 raft_consensus.cc:3037] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.616802 27723 raft_consensus.cc:2454] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.616928 27524 leader_election.cc:302] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.617002 28507 raft_consensus.cc:2781] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.617170 28507 raft_consensus.cc:686] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.617234 28507 consensus_queue.cc:237] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.618088 27198 catalog_manager.cc:5484] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.626402 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d6c135f8fc7a499bb9da105ad1f543c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.626524 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6c135f8fc7a499bb9da105ad1f543c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.628345 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 036d2832bef249b3b350c8916297ffbb (DEFAULT_TABLE table=default.Table_d6c135f8fc7a499bb9da105ad1f543c2 [id=1364def75674409381028490b35c3208]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.628397 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 036d2832bef249b3b350c8916297ffbb (DEFAULT_TABLE table=default.Table_d6c135f8fc7a499bb9da105ad1f543c2 [id=1364def75674409381028490b35c3208]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.628465 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d2832bef249b3b350c8916297ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.628520 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d2832bef249b3b350c8916297ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.629314 28664 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.629365 28655 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.629489 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 036d2832bef249b3b350c8916297ffbb (DEFAULT_TABLE table=default.Table_d6c135f8fc7a499bb9da105ad1f543c2 [id=1364def75674409381028490b35c3208]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.629570 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d2832bef249b3b350c8916297ffbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.629731 28664 tablet_bootstrap.cc:654] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.629731 28655 tablet_bootstrap.cc:654] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.630267 28664 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.630317 28664 ts_tablet_manager.cc:1367] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.630342 28655 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.630378 28655 ts_tablet_manager.cc:1367] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.630483 28664 raft_consensus.cc:348] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.630545 28664 raft_consensus.cc:374] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.630534 28655 raft_consensus.cc:348] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.630569 28664 raft_consensus.cc:729] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.630587 28655 raft_consensus.cc:374] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.630604 28655 raft_consensus.cc:729] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.630607 28664 consensus_queue.cc:260] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.630641 28655 consensus_queue.cc:260] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.630700 28664 ts_tablet_manager.cc:1398] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:16.630730 28655 ts_tablet_manager.cc:1398] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.630903 28656 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.631282 28656 tablet_bootstrap.cc:654] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.632109 28656 tablet_bootstrap.cc:492] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.632158 28656 ts_tablet_manager.cc:1367] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.632282 28656 raft_consensus.cc:348] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.632335 28656 raft_consensus.cc:374] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.632352 28656 raft_consensus.cc:729] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.632390 28656 consensus_queue.cc:260] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.632481 28656 ts_tablet_manager.cc:1398] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.633286 28507 consensus_queue.cc:1035] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.637497 28767 raft_consensus.cc:482] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.637547 28767 raft_consensus.cc:504] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.637640 28767 leader_election.cc:288] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.637823 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d2832bef249b3b350c8916297ffbb" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.637884 27582 raft_consensus.cc:2454] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.637866 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d2832bef249b3b350c8916297ffbb" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.637916 27719 raft_consensus.cc:2454] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.638008 27386 leader_election.cc:302] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.637830 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_61db889c4c5a45e793a8bf5976fc5eac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.638078 28767 raft_consensus.cc:2781] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.638106 28767 raft_consensus.cc:482] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20240503 02:09:16.638103 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_61db889c4c5a45e793a8bf5976fc5eac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.638123 28767 raft_consensus.cc:3037] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.638758 28767 raft_consensus.cc:504] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.638864 28767 leader_election.cc:288] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.639088 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d2832bef249b3b350c8916297ffbb" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.639158 27582 raft_consensus.cc:3037] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.639184 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_07a533b683494296ad4496688fa5fc47" } modify_external_catalogs: true
I20240503 02:09:16.639199 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d2832bef249b3b350c8916297ffbb" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.639236 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_07a533b683494296ad4496688fa5fc47" } modify_external_catalogs: true
I20240503 02:09:16.639258 27723 raft_consensus.cc:3037] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.639793 27582 raft_consensus.cc:2454] T 036d2832bef249b3b350c8916297ffbb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.639899 27723 raft_consensus.cc:2454] T 036d2832bef249b3b350c8916297ffbb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.640033 29240 consensus_queue.cc:1035] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.640355 27386 leader_election.cc:302] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.640471 28685 raft_consensus.cc:2781] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.640514 28685 raft_consensus.cc:686] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.640547 28685 consensus_queue.cc:237] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.641959 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8825d358600d49cea73d51fa734e53ca
I20240503 02:09:16.642230 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 8825d358600d49cea73d51fa734e53ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.642285 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 8825d358600d49cea73d51fa734e53ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.642367 28654 tablet_replica.cc:330] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.642433 28654 raft_consensus.cc:2229] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.642495 28654 raft_consensus.cc:2258] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.642706 28654 ts_tablet_manager.cc:1875] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.643133 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_81906986fe914a47b647cdaffb6e0880"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.643173 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8825d358600d49cea73d51fa734e53ca with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
W20240503 02:09:16.643224 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81906986fe914a47b647cdaffb6e0880 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.643257 28653 tablet_replica.cc:330] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.643307 28653 raft_consensus.cc:2229] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.643337 28653 raft_consensus.cc:2258] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.643539 28653 ts_tablet_manager.cc:1875] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.644068 28654 ts_tablet_manager.cc:1888] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.644109 28654 log.cc:1196] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8825d358600d49cea73d51fa734e53ca
I20240503 02:09:16.644289 28654 ts_tablet_manager.cc:1909] T 8825d358600d49cea73d51fa734e53ca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.644604 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b2424d6c3f964a17b69baea107c5cde3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.644724 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2424d6c3f964a17b69baea107c5cde3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.644922 28653 ts_tablet_manager.cc:1888] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.644973 28653 log.cc:1196] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8825d358600d49cea73d51fa734e53ca
I20240503 02:09:16.645162 28653 ts_tablet_manager.cc:1909] T 8825d358600d49cea73d51fa734e53ca P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.645506 28767 consensus_queue.cc:1035] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.645795 27703 tablet_service.cc:1467] Processing CreateTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.645903 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5377c9b1e6c419e8f8a3ed4f0855c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.645895 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.645984 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5377c9b1e6c419e8f8a3ed4f0855c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.646957 28655 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.647204 27560 tablet_service.cc:1467] Processing CreateTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.647287 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5377c9b1e6c419e8f8a3ed4f0855c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.647336 28655 tablet_bootstrap.cc:654] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.647533 28664 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.647883 28664 tablet_bootstrap.cc:654] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.648488 28664 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.648535 28664 ts_tablet_manager.cc:1367] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.648651 28664 raft_consensus.cc:348] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.648705 28664 raft_consensus.cc:374] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.648722 28664 raft_consensus.cc:729] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.648754 28664 consensus_queue.cc:260] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.648839 28664 ts_tablet_manager.cc:1398] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.648882 28656 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.648996 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8825d358600d49cea73d51fa734e53ca (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:16.649221 28656 tablet_bootstrap.cc:654] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.649302 28661 tablet_replica.cc:330] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.649410 28661 raft_consensus.cc:2229] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.649436 28661 raft_consensus.cc:2258] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.649642 28661 ts_tablet_manager.cc:1875] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.650100 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8825d358600d49cea73d51fa734e53ca (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:16.650846 28661 ts_tablet_manager.cc:1888] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.650879 28661 log.cc:1196] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8825d358600d49cea73d51fa734e53ca
I20240503 02:09:16.650997 28661 ts_tablet_manager.cc:1909] T 8825d358600d49cea73d51fa734e53ca P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.651773 27198 catalog_manager.cc:5484] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.652601 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8825d358600d49cea73d51fa734e53ca (table default.Table_07a533b683494296ad4496688fa5fc47 [id=726578606f4b43acaa6898f9b234a41e]) successfully deleted
I20240503 02:09:16.652741 28656 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.652794 28656 ts_tablet_manager.cc:1367] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:16.652804 28655 tablet_bootstrap.cc:492] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.652848 28655 ts_tablet_manager.cc:1367] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:16.652906 28656 raft_consensus.cc:348] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.652962 28656 raft_consensus.cc:374] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.652983 28656 raft_consensus.cc:729] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.652993 28655 raft_consensus.cc:348] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.653028 28656 consensus_queue.cc:260] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.653055 28655 raft_consensus.cc:374] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.653079 28655 raft_consensus.cc:729] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.653107 28656 ts_tablet_manager.cc:1398] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.653112 28655 consensus_queue.cc:260] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.653195 28655 ts_tablet_manager.cc:1398] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.655097 28685 raft_consensus.cc:482] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.655148 28685 raft_consensus.cc:504] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.655234 28685 leader_election.cc:288] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.655332 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6d82db9bc94f95ad0e602a43bcdc87" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.655395 27582 raft_consensus.cc:2454] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.655486 27386 leader_election.cc:302] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.655556 28409 raft_consensus.cc:2781] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.655586 28409 raft_consensus.cc:482] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.655602 28409 raft_consensus.cc:3037] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.655854 28685 consensus_queue.cc:1035] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.656183 28409 raft_consensus.cc:504] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.656289 28409 leader_election.cc:288] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.656347 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6d82db9bc94f95ad0e602a43bcdc87" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.656397 27719 raft_consensus.cc:2454] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.657068 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6d82db9bc94f95ad0e602a43bcdc87" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.657130 27582 raft_consensus.cc:3037] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.657191 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f6d82db9bc94f95ad0e602a43bcdc87" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.657244 27719 raft_consensus.cc:3037] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.657683 27582 raft_consensus.cc:2454] T 1f6d82db9bc94f95ad0e602a43bcdc87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.657822 27719 raft_consensus.cc:2454] T 1f6d82db9bc94f95ad0e602a43bcdc87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.657976 27388 leader_election.cc:302] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.658063 28409 raft_consensus.cc:2781] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.658039 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8812ecdff34e469998f5419dcb07669b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.658113 28409 raft_consensus.cc:686] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
W20240503 02:09:16.658131 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8812ecdff34e469998f5419dcb07669b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.658146 28409 consensus_queue.cc:237] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.659353 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 1186037b246e4c70ac3196564d599c54 (DEFAULT_TABLE table=default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.659358 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 837debf700f74dc680eb88168aeac413 (DEFAULT_TABLE table=default.Table_81906986fe914a47b647cdaffb6e0880 [id=260be253b97d4a5681ac6d6875ad7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.659461 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1186037b246e4c70ac3196564d599c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.659463 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837debf700f74dc680eb88168aeac413. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.659704 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 837debf700f74dc680eb88168aeac413 (DEFAULT_TABLE table=default.Table_81906986fe914a47b647cdaffb6e0880 [id=260be253b97d4a5681ac6d6875ad7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.659752 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 1186037b246e4c70ac3196564d599c54 (DEFAULT_TABLE table=default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.659781 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837debf700f74dc680eb88168aeac413. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.659840 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1186037b246e4c70ac3196564d599c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.660540 28655 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.660581 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 1186037b246e4c70ac3196564d599c54 (DEFAULT_TABLE table=default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.660665 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1186037b246e4c70ac3196564d599c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.660784 28655 tablet_bootstrap.cc:654] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.660884 28656 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.661242 28656 tablet_bootstrap.cc:654] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.661347 28664 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.661645 28664 tablet_bootstrap.cc:654] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.659353 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 837debf700f74dc680eb88168aeac413 (DEFAULT_TABLE table=default.Table_81906986fe914a47b647cdaffb6e0880 [id=260be253b97d4a5681ac6d6875ad7414]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.661912 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837debf700f74dc680eb88168aeac413. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.661950 27196 catalog_manager.cc:5484] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.662400 28656 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.662436 28656 ts_tablet_manager.cc:1367] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.662552 28656 raft_consensus.cc:348] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.662609 28656 raft_consensus.cc:374] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.662626 28656 raft_consensus.cc:729] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.662663 28656 consensus_queue.cc:260] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.662752 28656 ts_tablet_manager.cc:1398] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.662765 28664 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.662803 28664 ts_tablet_manager.cc:1367] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.662803 28655 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.662837 28655 ts_tablet_manager.cc:1367] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.662932 28664 raft_consensus.cc:348] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.662982 28664 raft_consensus.cc:374] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.662967 28655 raft_consensus.cc:348] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.662999 28664 raft_consensus.cc:729] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.663029 28655 raft_consensus.cc:374] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.663054 28655 raft_consensus.cc:729] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.663048 28664 consensus_queue.cc:260] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.663095 28655 consensus_queue.cc:260] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.663130 28664 ts_tablet_manager.cc:1398] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.663179 28655 ts_tablet_manager.cc:1398] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.663205 28664 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.663236 28655 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.663540 28656 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.663559 28664 tablet_bootstrap.cc:654] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.663559 28655 tablet_bootstrap.cc:654] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.663868 28656 tablet_bootstrap.cc:654] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.664222 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_49dc43dbcbba437e8e7a442d4cda81b3" } new_table_name: "default.Table_2534db089d5f41f1af1f14203414381c" modify_external_catalogs: true
I20240503 02:09:16.664664 27582 raft_consensus.cc:1261] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.664710 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.664827 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.664799 28685 consensus_queue.cc:1035] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.664952 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5cd03ad74f1f45fb81b1d87830a025d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.665027 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cd03ad74f1f45fb81b1d87830a025d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.666507 27723 raft_consensus.cc:1261] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.666712 28409 consensus_queue.cc:1035] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.667124 28655 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.667160 28655 ts_tablet_manager.cc:1367] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.667279 28655 raft_consensus.cc:348] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.667313 28655 raft_consensus.cc:374] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.667325 28655 raft_consensus.cc:729] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.667364 28655 consensus_queue.cc:260] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.667455 28655 ts_tablet_manager.cc:1398] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.668659 27582 raft_consensus.cc:1261] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.668669 27453 raft_consensus.cc:1261] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.668808 28774 consensus_queue.cc:1035] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.669128 27560 tablet_service.cc:1467] Processing CreateTablet for tablet ee57ef1446e246ba98a59945fbcad76e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.669210 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee57ef1446e246ba98a59945fbcad76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.669692 27692 tablet_service.cc:1467] Processing CreateTablet for tablet ee57ef1446e246ba98a59945fbcad76e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.669826 28694 tablet.cc:1722] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.670336 27425 tablet_service.cc:1467] Processing CreateTablet for tablet ee57ef1446e246ba98a59945fbcad76e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.670414 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee57ef1446e246ba98a59945fbcad76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.670418 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee57ef1446e246ba98a59945fbcad76e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.670878 28693 consensus_queue.cc:1035] T a1177866c9e94ce9bfdc45ecbc6172d2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.671468 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.671774 28656 tablet_bootstrap.cc:492] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.671816 28656 ts_tablet_manager.cc:1367] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.671943 28656 raft_consensus.cc:348] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.671991 28656 raft_consensus.cc:374] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.672008 28656 raft_consensus.cc:729] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.672039 28656 consensus_queue.cc:260] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.672134 28656 ts_tablet_manager.cc:1398] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.672184 28656 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.672396 28710 tablet.cc:1722] T a1177866c9e94ce9bfdc45ecbc6172d2 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.672488 28656 tablet_bootstrap.cc:654] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.672549 28655 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.672624 28410 raft_consensus.cc:482] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.672662 28410 raft_consensus.cc:504] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.672751 28410 leader_election.cc:288] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.672957 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19e5d244a9364cd595a1089c421d9a06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.673024 27449 raft_consensus.cc:2454] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.673070 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 7d41e4e82931437e808ac39826c285b1 (DEFAULT_TABLE table=default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.673059 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19e5d244a9364cd595a1089c421d9a06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.673115 27719 raft_consensus.cc:2454] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.673147 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d41e4e82931437e808ac39826c285b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.673378 27523 leader_election.cc:302] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.673445 28507 raft_consensus.cc:2781] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.673473 28507 raft_consensus.cc:482] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.673491 28507 raft_consensus.cc:3037] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.674017 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e0f31cbd417c8cd13696c24f980c (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.674047 28507 raft_consensus.cc:504] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.674113 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 7d41e4e82931437e808ac39826c285b1 (DEFAULT_TABLE table=default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.674124 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e0f31cbd417c8cd13696c24f980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.674152 28507 leader_election.cc:288] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.674183 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d41e4e82931437e808ac39826c285b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.674327 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_2534db089d5f41f1af1f14203414381c [id=89289d6ec12b4857ab4449223dc00d62] alter complete (version 1)
I20240503 02:09:16.674551 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19e5d244a9364cd595a1089c421d9a06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.674605 27719 raft_consensus.cc:3037] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.674752 28664 tablet_bootstrap.cc:492] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.674789 28664 ts_tablet_manager.cc:1367] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:16.674922 28664 raft_consensus.cc:348] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.674970 28664 raft_consensus.cc:374] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.675005 28664 raft_consensus.cc:729] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.675052 28664 consensus_queue.cc:260] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.675134 28664 ts_tablet_manager.cc:1398] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.675190 27719 raft_consensus.cc:2454] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.675295 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e0f31cbd417c8cd13696c24f980c (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.675380 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e0f31cbd417c8cd13696c24f980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.675585 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19e5d244a9364cd595a1089c421d9a06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.675639 27449 raft_consensus.cc:3037] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.676143 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.676183 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a1177866c9e94ce9bfdc45ecbc6172d2 (table default.Table_2534db089d5f41f1af1f14203414381c [id=89289d6ec12b4857ab4449223dc00d62]). Expected version 1 got 0
I20240503 02:09:16.676257 27449 raft_consensus.cc:2454] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.676278 27523 leader_election.cc:302] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.676630 28686 tablet.cc:1722] T d20aa1acdfbf4c27ba2b5a5dbc75483d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.676795 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ed36149ca1e94fca90786efbe99c4cba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.676901 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed36149ca1e94fca90786efbe99c4cba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.677232 27587 raft_consensus.cc:1261] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.677244 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 1388e0f31cbd417c8cd13696c24f980c (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.677425 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1388e0f31cbd417c8cd13696c24f980c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.677460 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0d3720cfc5c141a4aec2a7fd638195d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.677541 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d3720cfc5c141a4aec2a7fd638195d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.678014 28766 consensus_queue.cc:1035] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.678522 27723 raft_consensus.cc:1261] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.679543 29240 raft_consensus.cc:2781] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.679591 29240 raft_consensus.cc:686] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.679651 29240 consensus_queue.cc:237] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.679682 29167 tablet.cc:1722] T d20aa1acdfbf4c27ba2b5a5dbc75483d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.679737 28767 consensus_queue.cc:1035] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.680135 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_66c2965235094c56a59bfb004135fc7a" } modify_external_catalogs: true
I20240503 02:09:16.680188 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_66c2965235094c56a59bfb004135fc7a" } modify_external_catalogs: true
I20240503 02:09:16.680330 28656 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.680366 28656 ts_tablet_manager.cc:1367] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.680492 28656 raft_consensus.cc:348] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.680552 28656 raft_consensus.cc:374] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.680569 28656 raft_consensus.cc:729] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.680542 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4d99d803f2ac4b599aea7043cceb1506" } new_table_name: "default.Table_49384112661b4fbba2fbfedd6927a068" modify_external_catalogs: true
I20240503 02:09:16.680603 28656 consensus_queue.cc:260] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.680685 28656 ts_tablet_manager.cc:1398] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.680732 28656 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.680770 28694 tablet.cc:1722] T d20aa1acdfbf4c27ba2b5a5dbc75483d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.681085 28656 tablet_bootstrap.cc:654] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.681452 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 7d41e4e82931437e808ac39826c285b1 (DEFAULT_TABLE table=default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.681525 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d41e4e82931437e808ac39826c285b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.681728 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 949b2d82e23340d3b4ce56934a4db493 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.681805 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949b2d82e23340d3b4ce56934a4db493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.682479 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 949b2d82e23340d3b4ce56934a4db493 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.682552 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949b2d82e23340d3b4ce56934a4db493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.683084 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet d20aa1acdfbf4c27ba2b5a5dbc75483d (table default.Table_8b7ea237bea9428f8653574227fc6436 [id=2c48df2e6d864607b534fa164f539bcd]). Expected version 2 got 0
I20240503 02:09:16.683791 28656 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.683827 28656 ts_tablet_manager.cc:1367] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.683956 28656 raft_consensus.cc:348] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.684007 28656 raft_consensus.cc:374] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.684024 28656 raft_consensus.cc:729] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.684056 28656 consensus_queue.cc:260] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.684142 28656 ts_tablet_manager.cc:1398] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.684187 28656 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.684494 28656 tablet_bootstrap.cc:654] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.684623 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.684880 28686 tablet.cc:1722] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.684957 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 327dc0c3209f454499caeb31cdbe3f5d from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.685348 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.685714 28655 tablet_bootstrap.cc:654] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.686262 28656 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.686297 28656 ts_tablet_manager.cc:1367] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:16.686416 28656 raft_consensus.cc:348] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.686465 28656 raft_consensus.cc:374] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.686481 28656 raft_consensus.cc:729] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.686523 28656 consensus_queue.cc:260] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.686610 28656 ts_tablet_manager.cc:1398] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.686658 28656 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.686908 29558 tablet.cc:1722] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.686920 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.686960 28656 tablet_bootstrap.cc:654] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.687001 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9b66e153765c42128c4f64f0894fbe87
I20240503 02:09:16.687245 28655 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.687283 28655 ts_tablet_manager.cc:1367] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240503 02:09:16.687368 28655 raft_consensus.cc:348] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.687417 28655 raft_consensus.cc:374] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.687433 28655 raft_consensus.cc:729] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.687449 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b66e153765c42128c4f64f0894fbe87 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.687466 28655 consensus_queue.cc:260] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.687556 28654 tablet_replica.cc:330] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.687582 28655 ts_tablet_manager.cc:1398] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.687605 28654 raft_consensus.cc:2229] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.687639 28655 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.687642 28654 raft_consensus.cc:2258] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.687454 29167 tablet.cc:1722] T a1177866c9e94ce9bfdc45ecbc6172d2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.687857 28655 tablet_bootstrap.cc:654] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.687898 28654 ts_tablet_manager.cc:1875] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.688060 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b66e153765c42128c4f64f0894fbe87 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.688151 28653 tablet_replica.cc:330] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.688210 28653 raft_consensus.cc:2229] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.688246 28653 raft_consensus.cc:2258] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.675190 28664 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.688916 28664 tablet_bootstrap.cc:654] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:16.689002 27195 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b02d8134f2684fb5a0ddbb9e685c8f79 consensus state has no leader
I20240503 02:09:16.689059 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 327dc0c3209f454499caeb31cdbe3f5d
I20240503 02:09:16.689114 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 327dc0c3209f454499caeb31cdbe3f5d deleted and replaced by tablet 910de4fd567b485f9ebaa9e338eeb114
I20240503 02:09:16.689189 28654 ts_tablet_manager.cc:1888] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.689230 28654 log.cc:1196] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9b66e153765c42128c4f64f0894fbe87
I20240503 02:09:16.689400 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b66e153765c42128c4f64f0894fbe87 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.689397 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d with delete_type TABLET_DATA_DELETED (replaced by tablet 910de4fd567b485f9ebaa9e338eeb114) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.689473 28661 tablet_replica.cc:330] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.689520 28661 raft_consensus.cc:2229] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.689571 28661 raft_consensus.cc:2258] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.689615 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d with delete_type TABLET_DATA_DELETED (replaced by tablet 910de4fd567b485f9ebaa9e338eeb114) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.689635 28653 ts_tablet_manager.cc:1875] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.689640 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a99054ca9753450db1ac75a35ae25e55 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.689718 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99054ca9753450db1ac75a35ae25e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.689750 28661 ts_tablet_manager.cc:1875] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.690588 27425 tablet_service.cc:1467] Processing CreateTablet for tablet a99054ca9753450db1ac75a35ae25e55 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.690694 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99054ca9753450db1ac75a35ae25e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.691020 28661 ts_tablet_manager.cc:1888] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.691062 28661 log.cc:1196] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9b66e153765c42128c4f64f0894fbe87
I20240503 02:09:16.691089 28655 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.691128 28655 ts_tablet_manager.cc:1367] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.691201 28655 raft_consensus.cc:348] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.691241 28661 ts_tablet_manager.cc:1909] T 9b66e153765c42128c4f64f0894fbe87 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.691253 28655 raft_consensus.cc:374] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.691272 28655 raft_consensus.cc:729] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.691306 28655 consensus_queue.cc:260] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.691390 28655 ts_tablet_manager.cc:1398] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.691445 28655 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.691666 28655 tablet_bootstrap.cc:654] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.691738 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_998ab3c38dc14137985b066bb7f5d8a4" } new_table_name: "default.Table_5f9ff9ff07104d2f8b1fc7bab7347d37" modify_external_catalogs: true
I20240503 02:09:16.691856 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_855451a819a547e7b30be257415059ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.691952 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_855451a819a547e7b30be257415059ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.692214 28653 ts_tablet_manager.cc:1888] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.692256 28653 log.cc:1196] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9b66e153765c42128c4f64f0894fbe87
I20240503 02:09:16.692431 28653 ts_tablet_manager.cc:1909] T 9b66e153765c42128c4f64f0894fbe87 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.692524 29046 mvcc.cc:205] Tried to move back new op lower bound from 7023420033712930816 to 7023420033250254848. Current Snapshot: MvccSnapshot[applied={T|T < 7023420033712930816}]
I20240503 02:09:16.692564 28698 tablet.cc:1722] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.692775 27692 tablet_service.cc:1467] Processing CreateTablet for tablet a99054ca9753450db1ac75a35ae25e55 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.692862 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a99054ca9753450db1ac75a35ae25e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.693281 28653 tablet_replica.cc:330] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.693373 28653 raft_consensus.cc:2229] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.693404 28653 raft_consensus.cc:2258] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.693574 27198 catalog_manager.cc:5484] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.693668 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
I20240503 02:09:16.693701 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:16.693706 28656 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.693742 28656 ts_tablet_manager.cc:1367] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:16.693833 28409 consensus_queue.cc:1035] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.693853 28656 raft_consensus.cc:348] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.693904 28656 raft_consensus.cc:374] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.693922 28656 raft_consensus.cc:729] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.693957 28656 consensus_queue.cc:260] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.694025 28653 ts_tablet_manager.cc:1875] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.694032 28656 ts_tablet_manager.cc:1398] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.694084 28656 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.694307 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:16.694406 28656 tablet_bootstrap.cc:654] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.694916 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.694998 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910de4fd567b485f9ebaa9e338eeb114. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.689420 28654 ts_tablet_manager.cc:1909] T 9b66e153765c42128c4f64f0894fbe87 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.695129 28653 ts_tablet_manager.cc:1888] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.695164 28653 log.cc:1196] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/327dc0c3209f454499caeb31cdbe3f5d
I20240503 02:09:16.695307 28654 tablet_replica.cc:330] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.695328 28653 ts_tablet_manager.cc:1909] T 327dc0c3209f454499caeb31cdbe3f5d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.695371 28654 raft_consensus.cc:2229] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.695407 28654 raft_consensus.cc:2258] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.695560 28654 ts_tablet_manager.cc:1875] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.696331 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 327dc0c3209f454499caeb31cdbe3f5d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.696501 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 327dc0c3209f454499caeb31cdbe3f5d with delete_type TABLET_DATA_DELETED (replaced by tablet 910de4fd567b485f9ebaa9e338eeb114) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.696609 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.696610 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_53dadd09f9a34fe0b1f8cdd2439b7a37" } new_table_name: "default.Table_38fa31d9bcc54033a6c87470b7d1de63" modify_external_catalogs: true
I20240503 02:09:16.696692 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910de4fd567b485f9ebaa9e338eeb114. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.697057 28661 tablet_replica.cc:330] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.697118 28661 raft_consensus.cc:2229] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.697178 28661 raft_consensus.cc:2258] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.697214 28654 ts_tablet_manager.cc:1888] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.697293 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:16.697252 28654 log.cc:1196] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/327dc0c3209f454499caeb31cdbe3f5d
I20240503 02:09:16.697494 27853 consensus_queue.cc:1035] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.697677 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9b66e153765c42128c4f64f0894fbe87 (table default.Table_66c2965235094c56a59bfb004135fc7a [id=2c71029cd0524dd0be80de93b1362841]) successfully deleted
I20240503 02:09:16.697727 28654 ts_tablet_manager.cc:1909] T 327dc0c3209f454499caeb31cdbe3f5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.697747 28686 tablet.cc:1722] T 78311555b20a42d9b2884445a9dcf925 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.697788 28661 ts_tablet_manager.cc:1875] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.698297 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 327dc0c3209f454499caeb31cdbe3f5d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.699404 29566 tablet.cc:1722] T 78311555b20a42d9b2884445a9dcf925 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.699435 28661 ts_tablet_manager.cc:1888] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.699479 28661 log.cc:1196] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/327dc0c3209f454499caeb31cdbe3f5d
I20240503 02:09:16.699662 28661 ts_tablet_manager.cc:1909] T 327dc0c3209f454499caeb31cdbe3f5d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.700250 28685 raft_consensus.cc:482] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.700297 28685 raft_consensus.cc:504] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.700387 28685 leader_election.cc:288] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.700579 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 (DEFAULT_TABLE table=default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.700582 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee57ef1446e246ba98a59945fbcad76e" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.700659 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910de4fd567b485f9ebaa9e338eeb114. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.700968 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee57ef1446e246ba98a59945fbcad76e" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.701045 27583 raft_consensus.cc:2454] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.701112 28774 consensus_queue.cc:1035] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:16.701572 27388 leader_election.cc:341] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:16.701738 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 327dc0c3209f454499caeb31cdbe3f5d (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.702682 28694 tablet.cc:1722] T 78311555b20a42d9b2884445a9dcf925 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.702800 28686 tablet.cc:1722] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.703804 27560 tablet_service.cc:1467] Processing CreateTablet for tablet b715e519091b49858b528eb83bf33875 (DEFAULT_TABLE table=default.Table_855451a819a547e7b30be257415059ed [id=873d98bf48ab463c94fa75e3d6a59b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.703912 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b715e519091b49858b528eb83bf33875. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.703974 28694 tablet.cc:1722] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.704134 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet ac78bd3c0c254bc9b46a11df891d2c5b from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.705214 27425 tablet_service.cc:1467] Processing CreateTablet for tablet b715e519091b49858b528eb83bf33875 (DEFAULT_TABLE table=default.Table_855451a819a547e7b30be257415059ed [id=873d98bf48ab463c94fa75e3d6a59b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.705302 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b715e519091b49858b528eb83bf33875. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.705392 27692 tablet_service.cc:1467] Processing CreateTablet for tablet b715e519091b49858b528eb83bf33875 (DEFAULT_TABLE table=default.Table_855451a819a547e7b30be257415059ed [id=873d98bf48ab463c94fa75e3d6a59b06]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.705462 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b715e519091b49858b528eb83bf33875. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.706395 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_5f9ff9ff07104d2f8b1fc7bab7347d37 [id=37de95f92d3c41d59eb50ebe5d4652c4] alter complete (version 1)
I20240503 02:09:16.706758 27386 leader_election.cc:302] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters: c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:16.706931 28767 consensus_queue.cc:1035] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.707022 28767 raft_consensus.cc:2781] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.707047 28767 raft_consensus.cc:482] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.707062 28767 raft_consensus.cc:3037] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.707626 28655 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.707686 28655 ts_tablet_manager.cc:1367] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:16.707711 28656 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.707742 28656 ts_tablet_manager.cc:1367] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20240503 02:09:16.707769 28655 raft_consensus.cc:348] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.707830 28655 raft_consensus.cc:374] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.707851 28655 raft_consensus.cc:729] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.707871 28656 raft_consensus.cc:348] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.707890 28655 consensus_queue.cc:260] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.707926 28656 raft_consensus.cc:374] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.707950 28656 raft_consensus.cc:729] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.707973 28655 ts_tablet_manager.cc:1398] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.707983 28656 consensus_queue.cc:260] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.708034 28655 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.708076 28656 ts_tablet_manager.cc:1398] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.708122 28656 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.708254 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ac78bd3c0c254bc9b46a11df891d2c5b
I20240503 02:09:16.708334 27198 catalog_manager.cc:6554] ReplaceTablet: tablet ac78bd3c0c254bc9b46a11df891d2c5b deleted and replaced by tablet d3de7994307949ed9eee04c19f9080e3
I20240503 02:09:16.708348 28655 tablet_bootstrap.cc:654] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.708420 28656 tablet_bootstrap.cc:654] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.709012 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b with delete_type TABLET_DATA_DELETED (replaced by tablet d3de7994307949ed9eee04c19f9080e3) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.709090 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 949b2d82e23340d3b4ce56934a4db493 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.709175 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 949b2d82e23340d3b4ce56934a4db493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.707628 28767 raft_consensus.cc:504] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.709455 28767 leader_election.cc:288] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.709501 28654 tablet_replica.cc:330] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.709558 28654 raft_consensus.cc:2229] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.709553 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b with delete_type TABLET_DATA_DELETED (replaced by tablet d3de7994307949ed9eee04c19f9080e3) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.709589 28654 raft_consensus.cc:2258] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.709618 28661 tablet_replica.cc:330] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.709664 28661 raft_consensus.cc:2229] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.709676 28654 ts_tablet_manager.cc:1875] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.709712 28661 raft_consensus.cc:2258] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.709818 28661 ts_tablet_manager.cc:1875] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.709928 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_79149e8d4e8348d1aa6a3d1e6c4340d3" } new_table_name: "default.Table_443f4d53ffca4f209904f7da04bed3be" modify_external_catalogs: true
I20240503 02:09:16.710151 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet ac78bd3c0c254bc9b46a11df891d2c5b with delete_type TABLET_DATA_DELETED (replaced by tablet d3de7994307949ed9eee04c19f9080e3) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.710542 28655 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.710577 28655 ts_tablet_manager.cc:1367] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.710700 28655 raft_consensus.cc:348] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.710749 28655 raft_consensus.cc:374] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.710765 28655 raft_consensus.cc:729] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.710794 28655 consensus_queue.cc:260] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.710817 28654 ts_tablet_manager.cc:1888] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.710861 28654 log.cc:1196] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ac78bd3c0c254bc9b46a11df891d2c5b
I20240503 02:09:16.710892 28655 ts_tablet_manager.cc:1398] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.710951 28655 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.710999 28661 ts_tablet_manager.cc:1888] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.711047 28661 log.cc:1196] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ac78bd3c0c254bc9b46a11df891d2c5b
I20240503 02:09:16.711067 28654 ts_tablet_manager.cc:1909] T ac78bd3c0c254bc9b46a11df891d2c5b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.711230 28661 ts_tablet_manager.cc:1909] T ac78bd3c0c254bc9b46a11df891d2c5b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.711248 28655 tablet_bootstrap.cc:654] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.711344 28653 tablet_replica.cc:330] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.711393 28653 raft_consensus.cc:2229] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.711426 28653 raft_consensus.cc:2258] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.711419 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee57ef1446e246ba98a59945fbcad76e" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.711474 27583 raft_consensus.cc:3037] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.711519 28653 ts_tablet_manager.cc:1875] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.711553 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ac78bd3c0c254bc9b46a11df891d2c5b (table default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]) successfully deleted
I20240503 02:09:16.711935 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_307ac144366a49cea3bc45bd999c9ee2" } modify_external_catalogs: true
I20240503 02:09:16.711992 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_307ac144366a49cea3bc45bd999c9ee2" } modify_external_catalogs: true
I20240503 02:09:16.712049 27583 raft_consensus.cc:2454] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.712172 27386 leader_election.cc:302] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.712244 28767 raft_consensus.cc:2781] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.712286 28767 raft_consensus.cc:686] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.712317 28767 consensus_queue.cc:237] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.712504 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ac78bd3c0c254bc9b46a11df891d2c5b (table default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]) successfully deleted
I20240503 02:09:16.712484 28766 consensus_queue.cc:1035] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.712659 28653 ts_tablet_manager.cc:1888] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.712699 28653 log.cc:1196] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ac78bd3c0c254bc9b46a11df891d2c5b
I20240503 02:09:16.712872 28653 ts_tablet_manager.cc:1909] T ac78bd3c0c254bc9b46a11df891d2c5b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.712944 28655 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.712975 28655 ts_tablet_manager.cc:1367] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.713099 28655 raft_consensus.cc:348] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.713148 28655 raft_consensus.cc:374] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.713165 28655 raft_consensus.cc:729] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.713191 28655 consensus_queue.cc:260] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.713274 28655 ts_tablet_manager.cc:1398] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.713449 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ac78bd3c0c254bc9b46a11df891d2c5b (table default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]) successfully deleted
I20240503 02:09:16.713577 29167 tablet.cc:1722] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.713641 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80] alter complete (version 1)
I20240503 02:09:16.713876 28664 tablet_bootstrap.cc:492] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.713917 28664 ts_tablet_manager.cc:1367] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20240503 02:09:16.714057 28664 raft_consensus.cc:348] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.714121 28664 raft_consensus.cc:374] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.714140 28664 raft_consensus.cc:729] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.714179 28664 consensus_queue.cc:260] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.714275 28664 ts_tablet_manager.cc:1398] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.714342 28664 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.714668 28664 tablet_bootstrap.cc:654] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.714673 27565 tablet_service.cc:1467] Processing CreateTablet for tablet d3de7994307949ed9eee04c19f9080e3 (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.714737 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f679cfe32cbc48c0b6c188b60c2b08aa
I20240503 02:09:16.714758 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3de7994307949ed9eee04c19f9080e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.715076 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.715153 27427 tablet_service.cc:1467] Processing CreateTablet for tablet d3de7994307949ed9eee04c19f9080e3 (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.715260 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3de7994307949ed9eee04c19f9080e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.715411 27692 tablet_service.cc:1467] Processing CreateTablet for tablet d3de7994307949ed9eee04c19f9080e3 (DEFAULT_TABLE table=default.Table_59cd63322a1a4147a3929468a00f9fa1 [id=a47b4fbf485a474d81799d74917f5d3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.715379 27195 catalog_manager.cc:5484] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.715487 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3de7994307949ed9eee04c19f9080e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.715629 27587 raft_consensus.cc:1261] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.715907 27853 consensus_queue.cc:1035] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.716305 28409 raft_consensus.cc:482] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.716344 28409 raft_consensus.cc:504] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.716439 28409 leader_election.cc:288] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.715158 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.716647 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02d8134f2684fb5a0ddbb9e685c8f79" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.716704 27722 raft_consensus.cc:2454] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.716792 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee57ef1446e246ba98a59945fbcad76e" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.716828 27719 raft_consensus.cc:3037] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.716904 27388 leader_election.cc:302] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.717005 28767 raft_consensus.cc:2781] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.717031 28767 raft_consensus.cc:482] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.717044 28767 raft_consensus.cc:3037] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.717373 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02d8134f2684fb5a0ddbb9e685c8f79" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.717448 27585 raft_consensus.cc:2454] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.717456 28661 tablet_replica.cc:330] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.717509 28661 raft_consensus.cc:2229] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.717540 28661 raft_consensus.cc:2258] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.717567 28767 raft_consensus.cc:504] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.717664 28767 leader_election.cc:288] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.717708 27853 raft_consensus.cc:482] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.717746 27853 raft_consensus.cc:504] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.717830 27853 leader_election.cc:288] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.717913 27453 raft_consensus.cc:1261] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.717926 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02d8134f2684fb5a0ddbb9e685c8f79" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.717972 27585 raft_consensus.cc:3037] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.718451 28656 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.718459 28655 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.718488 28656 ts_tablet_manager.cc:1367] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:16.718528 27585 raft_consensus.cc:2454] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.718621 27722 raft_consensus.cc:1261] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.718624 28656 raft_consensus.cc:348] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.718684 28656 raft_consensus.cc:374] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.718703 28656 raft_consensus.cc:729] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.718739 28656 consensus_queue.cc:260] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.718750 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02d8134f2684fb5a0ddbb9e685c8f79" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.718798 27720 raft_consensus.cc:3037] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.718806 28655 tablet_bootstrap.cc:654] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.718816 28656 ts_tablet_manager.cc:1398] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.718865 28656 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.719198 28656 tablet_bootstrap.cc:654] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.719256 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 26dbf9f67602403e8d0b1b2fb82acebc from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.719280 27386 leader_election.cc:302] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.719352 28766 raft_consensus.cc:2781] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.719367 27449 raft_consensus.cc:1261] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.719383 28766 raft_consensus.cc:686] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.719420 27853 consensus_queue.cc:1035] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.719455 28766 consensus_queue.cc:237] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.719501 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1186037b246e4c70ac3196564d599c54" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.719559 27453 raft_consensus.cc:2454] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.719712 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1186037b246e4c70ac3196564d599c54" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.719769 27585 raft_consensus.cc:2454] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.719826 28410 consensus_queue.cc:1035] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.719852 27656 leader_election.cc:302] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.719882 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet f679cfe32cbc48c0b6c188b60c2b08aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.719946 28654 tablet_replica.cc:330] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.719992 28654 raft_consensus.cc:2229] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.720052 28654 raft_consensus.cc:2258] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.719964 28410 consensus_queue.cc:1035] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.720243 28654 ts_tablet_manager.cc:1875] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.720285 27853 raft_consensus.cc:2781] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.720314 27853 raft_consensus.cc:482] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.720331 27853 raft_consensus.cc:3037] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.720726 28661 ts_tablet_manager.cc:1875] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.721074 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
I20240503 02:09:16.721117 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 010d95e45f6e417c95d4d3e6278f4d74 (table default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7]). Expected version 1 got 0
I20240503 02:09:16.721143 27202 catalog_manager.cc:5484] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.721210 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_49384112661b4fbba2fbfedd6927a068 [id=09f75c45684343a9bae908f60652e20e]). Expected version 1 got 0
I20240503 02:09:16.721284 28655 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.721359 28655 ts_tablet_manager.cc:1367] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:16.721443 28654 ts_tablet_manager.cc:1888] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.721485 28654 log.cc:1196] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f679cfe32cbc48c0b6c188b60c2b08aa
I20240503 02:09:16.721488 28655 raft_consensus.cc:348] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.721539 28655 raft_consensus.cc:374] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.721554 28655 raft_consensus.cc:729] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.721586 28655 consensus_queue.cc:260] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.721657 28654 ts_tablet_manager.cc:1909] T f679cfe32cbc48c0b6c188b60c2b08aa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.721659 28655 ts_tablet_manager.cc:1398] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.721712 28655 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.722018 28661 ts_tablet_manager.cc:1888] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.722051 28655 tablet_bootstrap.cc:654] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.722061 28661 log.cc:1196] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f679cfe32cbc48c0b6c188b60c2b08aa
I20240503 02:09:16.722239 28661 ts_tablet_manager.cc:1909] T f679cfe32cbc48c0b6c188b60c2b08aa P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.722384 27719 raft_consensus.cc:2454] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.722596 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 26dbf9f67602403e8d0b1b2fb82acebc
I20240503 02:09:16.722676 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 26dbf9f67602403e8d0b1b2fb82acebc deleted and replaced by tablet e91b8d143d5c49c19d4a328e714b572d
I20240503 02:09:16.722688 28767 consensus_queue.cc:1035] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.723001 28653 tablet_replica.cc:330] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.723048 27720 raft_consensus.cc:2454] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.723066 28653 raft_consensus.cc:2229] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.723104 28653 raft_consensus.cc:2258] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.723116 28656 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.723155 28656 ts_tablet_manager.cc:1367] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.723273 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f679cfe32cbc48c0b6c188b60c2b08aa (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:16.723273 28656 raft_consensus.cc:348] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.723326 28656 raft_consensus.cc:374] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.723345 28656 raft_consensus.cc:729] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.723359 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bf8b557796624a6b8a7cce5936afa6ab" } modify_external_catalogs: true
I20240503 02:09:16.723377 28656 consensus_queue.cc:260] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.723400 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bf8b557796624a6b8a7cce5936afa6ab" } modify_external_catalogs: true
I20240503 02:09:16.723459 28656 ts_tablet_manager.cc:1398] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.723456 28653 ts_tablet_manager.cc:1875] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.723524 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc with delete_type TABLET_DATA_DELETED (replaced by tablet e91b8d143d5c49c19d4a328e714b572d) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.723680 27853 raft_consensus.cc:504] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.723780 27853 leader_election.cc:288] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.724380 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 010d95e45f6e417c95d4d3e6278f4d74 (table default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7]). Expected version 1 got 0
I20240503 02:09:16.724779 28653 ts_tablet_manager.cc:1888] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.724825 28653 log.cc:1196] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f679cfe32cbc48c0b6c188b60c2b08aa
I20240503 02:09:16.725000 28653 ts_tablet_manager.cc:1909] T f679cfe32cbc48c0b6c188b60c2b08aa P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.725220 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 19e5d244a9364cd595a1089c421d9a06 (table default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4]). Expected version 1 got 0
I20240503 02:09:16.725594 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc with delete_type TABLET_DATA_DELETED (replaced by tablet e91b8d143d5c49c19d4a328e714b572d) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.725631 28409 consensus_queue.cc:1035] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.725724 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 26dbf9f67602403e8d0b1b2fb82acebc with delete_type TABLET_DATA_DELETED (replaced by tablet e91b8d143d5c49c19d4a328e714b572d) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.725772 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f679cfe32cbc48c0b6c188b60c2b08aa (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:16.726146 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1186037b246e4c70ac3196564d599c54" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.726199 27453 raft_consensus.cc:3037] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.726441 29558 tablet.cc:1722] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.726553 28654 tablet_replica.cc:330] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.726611 28654 raft_consensus.cc:2229] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.726655 28654 pending_rounds.cc:62] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:16.726670 28654 pending_rounds.cc:68] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420031695683584 op_type: NO_OP noop_request { }
I20240503 02:09:16.726699 28654 raft_consensus.cc:2866] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.726718 28654 raft_consensus.cc:2258] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.726809 27453 raft_consensus.cc:2454] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.726917 28654 ts_tablet_manager.cc:1875] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.727142 27560 tablet_service.cc:1467] Processing CreateTablet for tablet e91b8d143d5c49c19d4a328e714b572d (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.727232 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91b8d143d5c49c19d4a328e714b572d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.727497 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1186037b246e4c70ac3196564d599c54" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.727573 27585 raft_consensus.cc:3037] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.728109 28654 ts_tablet_manager.cc:1888] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.728165 28654 log.cc:1196] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/26dbf9f67602403e8d0b1b2fb82acebc
I20240503 02:09:16.728360 28410 raft_consensus.cc:482] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.728423 28410 raft_consensus.cc:504] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.728513 28410 leader_election.cc:288] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.728572 28656 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.728910 28656 tablet_bootstrap.cc:654] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.728994 27659 leader_election.cc:302] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.728983 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5377c9b1e6c419e8f8a3ed4f0855c06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.729036 27450 raft_consensus.cc:2454] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.729090 28774 raft_consensus.cc:2781] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.729127 28774 raft_consensus.cc:686] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.729161 28774 consensus_queue.cc:237] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.729207 27692 tablet_service.cc:1467] Processing CreateTablet for tablet e91b8d143d5c49c19d4a328e714b572d (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.729269 27425 tablet_service.cc:1467] Processing CreateTablet for tablet e91b8d143d5c49c19d4a328e714b572d (DEFAULT_TABLE table=default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.729292 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91b8d143d5c49c19d4a328e714b572d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.729369 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91b8d143d5c49c19d4a328e714b572d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.729645 28774 consensus_queue.cc:1035] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.729979 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5377c9b1e6c419e8f8a3ed4f0855c06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.730032 27719 raft_consensus.cc:2454] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.730150 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 910de4fd567b485f9ebaa9e338eeb114
I20240503 02:09:16.730269 28661 tablet_replica.cc:330] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.730322 28661 raft_consensus.cc:2229] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.730348 28661 raft_consensus.cc:2258] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.730432 28661 ts_tablet_manager.cc:1875] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.730432 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7] alter complete (version 1)
I20240503 02:09:16.730554 27523 leader_election.cc:302] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.730571 28710 tablet.cc:1722] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.730621 28507 raft_consensus.cc:2781] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.730644 28507 raft_consensus.cc:482] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.730646 29090 mvcc.cc:205] Tried to move back new op lower bound from 7023420033905954816 to 7023420033399324672. Current Snapshot: MvccSnapshot[applied={T|T < 7023420033905954816}]
I20240503 02:09:16.730660 28507 raft_consensus.cc:3037] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.731012 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.731218 28507 raft_consensus.cc:504] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.731298 28656 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.731321 28507 leader_election.cc:288] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.731509 28661 ts_tablet_manager.cc:1888] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.731561 28661 log.cc:1196] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/26dbf9f67602403e8d0b1b2fb82acebc
I20240503 02:09:16.731745 28507 consensus_queue.cc:1035] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.732228 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f679cfe32cbc48c0b6c188b60c2b08aa (table default.Table_307ac144366a49cea3bc45bd999c9ee2 [id=d82d61ae21224a3a8b482ba096af77d9]) successfully deleted
I20240503 02:09:16.732311 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5377c9b1e6c419e8f8a3ed4f0855c06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.732362 27450 raft_consensus.cc:3037] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.732316 27195 catalog_manager.cc:5484] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.732436 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.732564 28661 ts_tablet_manager.cc:1909] T 26dbf9f67602403e8d0b1b2fb82acebc P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.732856 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 26dbf9f67602403e8d0b1b2fb82acebc (table default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]) successfully deleted
I20240503 02:09:16.732869 29551 consensus_queue.cc:1035] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.732914 27450 raft_consensus.cc:2454] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.728354 28654 ts_tablet_manager.cc:1909] T 26dbf9f67602403e8d0b1b2fb82acebc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
W20240503 02:09:16.733100 27189 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 910de4fd567b485f9ebaa9e338eeb114 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 910de4fd567b485f9ebaa9e338eeb114 already in progress: creating tablet
I20240503 02:09:16.733253 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5377c9b1e6c419e8f8a3ed4f0855c06" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.733284 28654 tablet_replica.cc:330] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.733345 28654 raft_consensus.cc:2229] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.733309 27719 raft_consensus.cc:3037] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.733372 28654 raft_consensus.cc:2258] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.733423 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 26dbf9f67602403e8d0b1b2fb82acebc (table default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]) successfully deleted
I20240503 02:09:16.733487 27524 leader_election.cc:302] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.733564 28507 raft_consensus.cc:2781] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.733611 28507 raft_consensus.cc:686] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.733644 28507 consensus_queue.cc:237] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.731334 28656 ts_tablet_manager.cc:1367] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:16.733922 27719 raft_consensus.cc:2454] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.734045 28656 raft_consensus.cc:348] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.734097 28656 raft_consensus.cc:374] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.734112 28656 raft_consensus.cc:729] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.734148 28656 consensus_queue.cc:260] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.734234 28656 ts_tablet_manager.cc:1398] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.734504 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.728106 27585 raft_consensus.cc:2454] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.734943 27198 catalog_manager.cc:5484] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.735019 28654 ts_tablet_manager.cc:1875] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.735119 28653 tablet_replica.cc:330] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.735172 28653 raft_consensus.cc:2229] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.735201 28653 raft_consensus.cc:2258] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.735288 28653 ts_tablet_manager.cc:1875] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.735554 28693 consensus_queue.cc:1035] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.737943 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 34c4d03198554e4aa7a11125f851debc from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.738304 28653 ts_tablet_manager.cc:1888] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.738422 28653 log.cc:1196] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/26dbf9f67602403e8d0b1b2fb82acebc
I20240503 02:09:16.738466 28654 ts_tablet_manager.cc:1888] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.738561 28654 log.cc:1196] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/910de4fd567b485f9ebaa9e338eeb114
I20240503 02:09:16.738620 28653 ts_tablet_manager.cc:1909] T 26dbf9f67602403e8d0b1b2fb82acebc P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.738766 28654 ts_tablet_manager.cc:1909] T 910de4fd567b485f9ebaa9e338eeb114 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.739079 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 26dbf9f67602403e8d0b1b2fb82acebc (table default.Table_d4ff0e8769f94e43a73d0dde16f9e531 [id=84032206f96a4a4e81537dbc9c69fba6]) successfully deleted
I20240503 02:09:16.739161 28653 tablet_replica.cc:330] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.739238 28653 raft_consensus.cc:2229] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.739274 28653 raft_consensus.cc:2258] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.739527 28653 ts_tablet_manager.cc:1875] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.739686 29167 tablet.cc:1722] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.739836 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 8fb685fcf4a045f99b2cdebe5bf164d7 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.740097 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 34c4d03198554e4aa7a11125f851debc
I20240503 02:09:16.740180 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 34c4d03198554e4aa7a11125f851debc deleted and replaced by tablet 6873697085ec4322a5871dd591dd81dd
I20240503 02:09:16.740324 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 34c4d03198554e4aa7a11125f851debc with delete_type TABLET_DATA_DELETED (replaced by tablet 6873697085ec4322a5871dd591dd81dd) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.740370 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 34c4d03198554e4aa7a11125f851debc with delete_type TABLET_DATA_DELETED (replaced by tablet 6873697085ec4322a5871dd591dd81dd) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.740473 28656 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.740793 28656 tablet_bootstrap.cc:654] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.741089 28653 ts_tablet_manager.cc:1888] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.741158 28653 log.cc:1196] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/910de4fd567b485f9ebaa9e338eeb114
I20240503 02:09:16.741220 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 34c4d03198554e4aa7a11125f851debc with delete_type TABLET_DATA_DELETED (replaced by tablet 6873697085ec4322a5871dd591dd81dd) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.741317 28653 ts_tablet_manager.cc:1909] T 910de4fd567b485f9ebaa9e338eeb114 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.741642 28661 tablet_replica.cc:330] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.741642 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 910de4fd567b485f9ebaa9e338eeb114 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.741676 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8fb685fcf4a045f99b2cdebe5bf164d7
I20240503 02:09:16.741705 28661 raft_consensus.cc:2229] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.741714 28654 tablet_replica.cc:330] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.741756 27202 catalog_manager.cc:6554] ReplaceTablet: tablet 8fb685fcf4a045f99b2cdebe5bf164d7 deleted and replaced by tablet 9181c05524c3474282c7f526e1195d29
I20240503 02:09:16.741758 28661 raft_consensus.cc:2258] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.741757 28654 raft_consensus.cc:2229] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.741791 28654 raft_consensus.cc:2258] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.741887 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 with delete_type TABLET_DATA_DELETED (replaced by tablet 9181c05524c3474282c7f526e1195d29) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.742008 28653 tablet_replica.cc:330] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.742060 28653 raft_consensus.cc:2229] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.742075 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 910de4fd567b485f9ebaa9e338eeb114 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.742094 28653 raft_consensus.cc:2258] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.742224 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 with delete_type TABLET_DATA_DELETED (replaced by tablet 9181c05524c3474282c7f526e1195d29) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.742267 28653 ts_tablet_manager.cc:1875] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.742300 28661 ts_tablet_manager.cc:1875] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.742627 28654 ts_tablet_manager.cc:1875] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.742655 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 6873697085ec4322a5871dd591dd81dd (DEFAULT_TABLE table=default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.742739 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6873697085ec4322a5871dd591dd81dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.743064 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 6873697085ec4322a5871dd591dd81dd (DEFAULT_TABLE table=default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.743229 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6873697085ec4322a5871dd591dd81dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.743384 28661 ts_tablet_manager.cc:1888] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.743425 28661 log.cc:1196] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/34c4d03198554e4aa7a11125f851debc
I20240503 02:09:16.743593 28661 ts_tablet_manager.cc:1909] T 34c4d03198554e4aa7a11125f851debc P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.743865 28661 tablet_replica.cc:330] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.743893 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 6873697085ec4322a5871dd591dd81dd (DEFAULT_TABLE table=default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.743921 28661 raft_consensus.cc:2229] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.743922 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 34c4d03198554e4aa7a11125f851debc (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]) successfully deleted
I20240503 02:09:16.743949 28661 raft_consensus.cc:2258] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.743975 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6873697085ec4322a5871dd591dd81dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.744902 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fb685fcf4a045f99b2cdebe5bf164d7 with delete_type TABLET_DATA_DELETED (replaced by tablet 9181c05524c3474282c7f526e1195d29) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.744941 28653 ts_tablet_manager.cc:1888] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.745055 28653 log.cc:1196] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/34c4d03198554e4aa7a11125f851debc
I20240503 02:09:16.745064 28654 ts_tablet_manager.cc:1888] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.745101 28654 log.cc:1196] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/34c4d03198554e4aa7a11125f851debc
I20240503 02:09:16.745779 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 9181c05524c3474282c7f526e1195d29 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.745867 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9181c05524c3474282c7f526e1195d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.746299 28653 ts_tablet_manager.cc:1909] T 34c4d03198554e4aa7a11125f851debc P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.746500 28653 tablet_replica.cc:330] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.746877 28653 raft_consensus.cc:2229] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.746942 28653 raft_consensus.cc:2258] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.747084 28655 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.747129 28655 ts_tablet_manager.cc:1367] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240503 02:09:16.747107 28654 ts_tablet_manager.cc:1909] T 34c4d03198554e4aa7a11125f851debc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.747268 28653 ts_tablet_manager.cc:1875] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.747251 28655 raft_consensus.cc:348] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.747327 28655 raft_consensus.cc:374] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.747351 28655 raft_consensus.cc:729] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.747385 28655 consensus_queue.cc:260] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.747471 28655 ts_tablet_manager.cc:1398] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.747560 28655 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.747843 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 9181c05524c3474282c7f526e1195d29 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.747875 28655 tablet_bootstrap.cc:654] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.747906 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 34c4d03198554e4aa7a11125f851debc (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]) successfully deleted
I20240503 02:09:16.747931 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9181c05524c3474282c7f526e1195d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.748039 28654 tablet_replica.cc:330] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.748117 28654 raft_consensus.cc:2229] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.748183 28654 raft_consensus.cc:2258] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.748268 28664 tablet_bootstrap.cc:492] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.748315 28664 ts_tablet_manager.cc:1367] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.000s
I20240503 02:09:16.748440 28654 ts_tablet_manager.cc:1875] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.748438 28664 raft_consensus.cc:348] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.748489 28664 raft_consensus.cc:374] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.748514 28664 raft_consensus.cc:729] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.748548 28664 consensus_queue.cc:260] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.748641 28664 ts_tablet_manager.cc:1398] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.748715 28664 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.748948 28653 ts_tablet_manager.cc:1888] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.748991 28653 log.cc:1196] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8fb685fcf4a045f99b2cdebe5bf164d7
I20240503 02:09:16.749042 28664 tablet_bootstrap.cc:654] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.749178 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 34c4d03198554e4aa7a11125f851debc (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]) successfully deleted
I20240503 02:09:16.749579 28410 consensus_queue.cc:1035] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.749959 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 9181c05524c3474282c7f526e1195d29 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.750072 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9181c05524c3474282c7f526e1195d29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.750353 28654 ts_tablet_manager.cc:1888] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.750403 28654 log.cc:1196] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8fb685fcf4a045f99b2cdebe5bf164d7
I20240503 02:09:16.750813 28654 ts_tablet_manager.cc:1909] T 8fb685fcf4a045f99b2cdebe5bf164d7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.747875 28767 consensus_queue.cc:1035] T ef574442d0ca4988b77635dd7e8ce5db P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20240503 02:09:16.751240 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:16.751319 28653 ts_tablet_manager.cc:1909] T 8fb685fcf4a045f99b2cdebe5bf164d7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.751571 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:16.752249 28656 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.752291 28656 ts_tablet_manager.cc:1367] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:16.752408 28656 raft_consensus.cc:348] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.752463 28656 raft_consensus.cc:374] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.752480 28656 raft_consensus.cc:729] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.752516 28656 consensus_queue.cc:260] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.752615 28656 ts_tablet_manager.cc:1398] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.752694 28656 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:16.752735 29586 consensus_queue.cc:1159] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c2be543b8aad4e80a58d6b4ab669faae" status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20240503 02:09:16.753019 28656 tablet_bootstrap.cc:654] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.753091 28661 ts_tablet_manager.cc:1875] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.753530 29167 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.753736 29558 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.754590 28661 ts_tablet_manager.cc:1888] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.754647 28661 log.cc:1196] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8fb685fcf4a045f99b2cdebe5bf164d7
I20240503 02:09:16.754809 28661 ts_tablet_manager.cc:1909] T 8fb685fcf4a045f99b2cdebe5bf164d7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.755681 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8fb685fcf4a045f99b2cdebe5bf164d7 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:16.755831 28710 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.757308 29550 consensus_queue.cc:1035] T 9fb59f7012094fab9cb807407fbc7996 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.758589 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e [id=80cf6485f989436d8fab264d6a5743b4] alter complete (version 1)
I20240503 02:09:16.759466 28664 tablet_bootstrap.cc:492] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.759513 28664 ts_tablet_manager.cc:1367] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:16.759650 28664 raft_consensus.cc:348] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.759698 28664 raft_consensus.cc:374] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.759716 28664 raft_consensus.cc:729] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.759748 28664 consensus_queue.cc:260] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.759737 29240 consensus_queue.cc:1035] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.759825 28664 ts_tablet_manager.cc:1398] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.759881 28664 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.760176 28664 tablet_bootstrap.cc:654] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.761433 28664 tablet_bootstrap.cc:492] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.761476 28664 ts_tablet_manager.cc:1367] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.761598 28664 raft_consensus.cc:348] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.761652 28664 raft_consensus.cc:374] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.761672 28664 raft_consensus.cc:729] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.761694 28664 consensus_queue.cc:260] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.761801 28664 ts_tablet_manager.cc:1398] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.761866 28664 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.762051 28656 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.762099 28656 ts_tablet_manager.cc:1367] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:16.762188 28664 tablet_bootstrap.cc:654] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.762220 28656 raft_consensus.cc:348] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.762288 28656 raft_consensus.cc:374] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.762313 28656 raft_consensus.cc:729] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.762352 28656 consensus_queue.cc:260] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.762444 28656 ts_tablet_manager.cc:1398] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.762519 28656 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.762580 28655 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.762615 28655 ts_tablet_manager.cc:1367] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20240503 02:09:16.762741 28655 raft_consensus.cc:348] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.762800 28655 raft_consensus.cc:374] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.762822 28655 raft_consensus.cc:729] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.762866 28656 tablet_bootstrap.cc:654] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.762859 28655 consensus_queue.cc:260] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.762955 28655 ts_tablet_manager.cc:1398] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.762985 28664 tablet_bootstrap.cc:492] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.763023 28664 ts_tablet_manager.cc:1367] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.763027 28655 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.763154 28664 raft_consensus.cc:348] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.763199 28664 raft_consensus.cc:374] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.763214 28664 raft_consensus.cc:729] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.763257 28664 consensus_queue.cc:260] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.763327 28664 ts_tablet_manager.cc:1398] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.763360 28655 tablet_bootstrap.cc:654] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.763368 28664 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.763841 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
W20240503 02:09:16.763911 27202 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6873697085ec4322a5871dd591dd81dd consensus state has no leader
I20240503 02:09:16.763676 28664 tablet_bootstrap.cc:654] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.764336 29240 consensus_queue.cc:1035] T d8424c39f615456e8db186a9c124629c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.764654 28685 consensus_queue.cc:1035] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.765483 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.769599 28655 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.769657 28656 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.769651 28655 ts_tablet_manager.cc:1367] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.769703 28656 ts_tablet_manager.cc:1367] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.769793 28655 raft_consensus.cc:348] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.769820 28656 raft_consensus.cc:348] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.769853 28655 raft_consensus.cc:374] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.769865 28656 raft_consensus.cc:374] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.769873 28655 raft_consensus.cc:729] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.769891 28656 raft_consensus.cc:729] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.769914 28655 consensus_queue.cc:260] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.769927 28656 consensus_queue.cc:260] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.769999 28656 ts_tablet_manager.cc:1398] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.770004 28655 ts_tablet_manager.cc:1398] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.770067 28655 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.770385 28655 tablet_bootstrap.cc:654] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.770704 28664 tablet_bootstrap.cc:492] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.770745 28664 ts_tablet_manager.cc:1367] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.770859 28664 raft_consensus.cc:348] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.770923 28664 raft_consensus.cc:374] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.770941 28664 raft_consensus.cc:729] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.770972 28664 consensus_queue.cc:260] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.771044 28664 ts_tablet_manager.cc:1398] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.771106 28664 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.771673 28664 tablet_bootstrap.cc:654] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.772325 29550 consensus_queue.cc:1035] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.772907 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 910de4fd567b485f9ebaa9e338eeb114 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.772980 28661 tablet_replica.cc:330] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.773027 28661 raft_consensus.cc:2229] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.773053 28661 raft_consensus.cc:2258] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.773886 28661 ts_tablet_manager.cc:1875] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.774202 28655 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.774240 28655 ts_tablet_manager.cc:1367] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:16.774354 28655 raft_consensus.cc:348] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.774399 28655 raft_consensus.cc:374] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.774416 28655 raft_consensus.cc:729] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.774464 28655 consensus_queue.cc:260] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.774542 28655 ts_tablet_manager.cc:1398] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.774610 28655 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.774920 28655 tablet_bootstrap.cc:654] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.775048 28661 ts_tablet_manager.cc:1888] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.775082 28661 log.cc:1196] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/910de4fd567b485f9ebaa9e338eeb114
I20240503 02:09:16.775197 28661 ts_tablet_manager.cc:1909] T 910de4fd567b485f9ebaa9e338eeb114 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.775487 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.775630 28664 tablet_bootstrap.cc:492] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.775673 28664 ts_tablet_manager.cc:1367] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:16.775794 28664 raft_consensus.cc:348] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.775857 28664 raft_consensus.cc:374] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.775882 28664 raft_consensus.cc:729] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.775924 28664 consensus_queue.cc:260] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.775990 28655 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.776012 28664 ts_tablet_manager.cc:1398] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.776027 28655 ts_tablet_manager.cc:1367] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.776118 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 910de4fd567b485f9ebaa9e338eeb114 (table default.Table_bf8b557796624a6b8a7cce5936afa6ab [id=3911408b6f2a4f6d8ce036f0a991f835]) successfully deleted
I20240503 02:09:16.776144 28655 raft_consensus.cc:348] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.776202 28655 raft_consensus.cc:374] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.776225 28655 raft_consensus.cc:729] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.776262 28655 consensus_queue.cc:260] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.776331 28664 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.776365 28655 ts_tablet_manager.cc:1398] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.776654 28664 tablet_bootstrap.cc:654] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.777031 27583 raft_consensus.cc:1261] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.777081 27718 raft_consensus.cc:1261] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.777288 28409 consensus_queue.cc:1035] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.777410 28409 consensus_queue.cc:1035] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.778056 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.778450 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_49384112661b4fbba2fbfedd6927a068 [id=09f75c45684343a9bae908f60652e20e]). Expected version 1 got 0
I20240503 02:09:16.779217 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_49384112661b4fbba2fbfedd6927a068 [id=09f75c45684343a9bae908f60652e20e]). Expected version 1 got 0
I20240503 02:09:16.779268 28710 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.779871 29558 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.781275 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_49384112661b4fbba2fbfedd6927a068 [id=09f75c45684343a9bae908f60652e20e] alter complete (version 1)
I20240503 02:09:16.783200 28664 tablet_bootstrap.cc:492] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.783243 28664 ts_tablet_manager.cc:1367] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.783393 28664 raft_consensus.cc:348] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.783694 28664 raft_consensus.cc:374] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.783725 28664 raft_consensus.cc:729] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.783862 28664 consensus_queue.cc:260] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.784014 28664 ts_tablet_manager.cc:1398] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.784065 28664 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.784171 29167 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.785611 28664 tablet_bootstrap.cc:654] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.788906 27583 raft_consensus.cc:3037] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.789536 28409 consensus_queue.cc:1035] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.789628 28664 tablet_bootstrap.cc:492] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.789669 28664 ts_tablet_manager.cc:1367] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:16.789942 28664 raft_consensus.cc:348] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.790115 28664 raft_consensus.cc:374] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.790144 28664 raft_consensus.cc:729] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.790190 28664 consensus_queue.cc:260] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.790513 28664 ts_tablet_manager.cc:1398] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.790588 28664 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.791165 28664 tablet_bootstrap.cc:654] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.792100 28664 tablet_bootstrap.cc:492] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.792138 28664 ts_tablet_manager.cc:1367] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.792253 28664 raft_consensus.cc:348] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.792294 28664 raft_consensus.cc:374] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.792309 28664 raft_consensus.cc:729] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.792347 28664 consensus_queue.cc:260] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.792428 28664 ts_tablet_manager.cc:1398] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.792569 28664 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.792898 28664 tablet_bootstrap.cc:654] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.793313 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.793597 28664 tablet_bootstrap.cc:492] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.793637 28664 ts_tablet_manager.cc:1367] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.793757 28664 raft_consensus.cc:348] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.793813 28664 raft_consensus.cc:374] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.793830 28664 raft_consensus.cc:729] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.793860 28664 consensus_queue.cc:260] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.793936 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.793947 28664 ts_tablet_manager.cc:1398] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.795439 28409 consensus_queue.cc:1035] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.797924 28774 raft_consensus.cc:482] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.797971 28774 raft_consensus.cc:504] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.798077 28774 leader_election.cc:288] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.798182 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99054ca9753450db1ac75a35ae25e55" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.798209 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99054ca9753450db1ac75a35ae25e55" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.798260 27583 raft_consensus.cc:2454] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.798260 27449 raft_consensus.cc:2454] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.798380 27656 leader_election.cc:302] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.798486 28774 raft_consensus.cc:2781] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.798518 28774 raft_consensus.cc:482] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.798530 28774 raft_consensus.cc:3037] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.799021 28774 raft_consensus.cc:504] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.799115 28774 leader_election.cc:288] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.799207 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99054ca9753450db1ac75a35ae25e55" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.799260 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a99054ca9753450db1ac75a35ae25e55" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.799283 27449 raft_consensus.cc:3037] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.799302 27583 raft_consensus.cc:3037] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.799746 27449 raft_consensus.cc:2454] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.799749 27583 raft_consensus.cc:2454] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.799912 27659 leader_election.cc:302] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.800071 28774 raft_consensus.cc:2781] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.800117 28774 raft_consensus.cc:686] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.800149 28774 consensus_queue.cc:237] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.801081 27198 catalog_manager.cc:5484] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.813536 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1535f39a27964b7e8dc4b587fc88a2b1" } modify_external_catalogs: true
I20240503 02:09:16.813602 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1535f39a27964b7e8dc4b587fc88a2b1" } modify_external_catalogs: true
I20240503 02:09:16.814450 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 5c89b472abbe4fceb027714afa92a7b9
I20240503 02:09:16.814664 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.814671 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.814693 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c89b472abbe4fceb027714afa92a7b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.814741 28653 tablet_replica.cc:330] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.814743 28654 tablet_replica.cc:330] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.814754 28661 tablet_replica.cc:330] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.814791 28661 raft_consensus.cc:2229] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.814790 28654 raft_consensus.cc:2229] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.814802 28653 raft_consensus.cc:2229] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.814819 28654 raft_consensus.cc:2258] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.814821 28661 raft_consensus.cc:2258] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.814848 28653 raft_consensus.cc:2258] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.814997 28654 ts_tablet_manager.cc:1875] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.815047 28653 ts_tablet_manager.cc:1875] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.815927 28661 ts_tablet_manager.cc:1875] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.816025 28654 ts_tablet_manager.cc:1888] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.816068 28653 ts_tablet_manager.cc:1888] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.816071 28654 log.cc:1196] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/5c89b472abbe4fceb027714afa92a7b9
I20240503 02:09:16.816109 28653 log.cc:1196] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/5c89b472abbe4fceb027714afa92a7b9
I20240503 02:09:16.816256 28653 ts_tablet_manager.cc:1909] T 5c89b472abbe4fceb027714afa92a7b9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.816277 28654 ts_tablet_manager.cc:1909] T 5c89b472abbe4fceb027714afa92a7b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.816589 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 5c89b472abbe4fceb027714afa92a7b9 (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:16.816594 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 5c89b472abbe4fceb027714afa92a7b9 (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:16.817045 28661 ts_tablet_manager.cc:1888] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.817090 28661 log.cc:1196] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/5c89b472abbe4fceb027714afa92a7b9
I20240503 02:09:16.817227 28661 ts_tablet_manager.cc:1909] T 5c89b472abbe4fceb027714afa92a7b9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.817553 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 5c89b472abbe4fceb027714afa92a7b9 (table default.Table_1535f39a27964b7e8dc4b587fc88a2b1 [id=610c0ff92ffd4fa2a73aee3b2a1041d7]) successfully deleted
I20240503 02:09:16.818120 28507 consensus_queue.cc:1035] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:16.818418 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:16.819406 28507 consensus_queue.cc:1035] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.827791 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3b492931b6ca49b096365669e265e21d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.827903 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b492931b6ca49b096365669e265e21d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.829843 27692 tablet_service.cc:1467] Processing CreateTablet for tablet fa67973d02924925a35b4418e1d4fa29 (DEFAULT_TABLE table=default.Table_3b492931b6ca49b096365669e265e21d [id=14ac8b9e93234f449dcf71726c9b7a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.829854 27565 tablet_service.cc:1467] Processing CreateTablet for tablet fa67973d02924925a35b4418e1d4fa29 (DEFAULT_TABLE table=default.Table_3b492931b6ca49b096365669e265e21d [id=14ac8b9e93234f449dcf71726c9b7a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.829845 27425 tablet_service.cc:1467] Processing CreateTablet for tablet fa67973d02924925a35b4418e1d4fa29 (DEFAULT_TABLE table=default.Table_3b492931b6ca49b096365669e265e21d [id=14ac8b9e93234f449dcf71726c9b7a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.829943 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa67973d02924925a35b4418e1d4fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.829943 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa67973d02924925a35b4418e1d4fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.829962 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa67973d02924925a35b4418e1d4fa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.831019 28664 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.831032 28655 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.831372 28655 tablet_bootstrap.cc:654] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.831394 28664 tablet_bootstrap.cc:654] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.831665 28656 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.831923 28655 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.831933 28656 tablet_bootstrap.cc:654] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.831964 28655 ts_tablet_manager.cc:1367] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.832127 28655 raft_consensus.cc:348] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.832181 28655 raft_consensus.cc:374] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.832199 28655 raft_consensus.cc:729] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.832232 28655 consensus_queue.cc:260] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.832321 28655 ts_tablet_manager.cc:1398] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.832345 28656 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.832384 28656 ts_tablet_manager.cc:1367] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.832460 28656 raft_consensus.cc:348] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.832500 28656 raft_consensus.cc:374] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.832515 28656 raft_consensus.cc:729] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.832557 28656 consensus_queue.cc:260] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.832629 28656 ts_tablet_manager.cc:1398] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.832818 28664 tablet_bootstrap.cc:492] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.832864 28664 ts_tablet_manager.cc:1367] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.832994 28664 raft_consensus.cc:348] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.833050 28664 raft_consensus.cc:374] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.833079 28664 raft_consensus.cc:729] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.833112 28664 consensus_queue.cc:260] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.833189 28664 ts_tablet_manager.cc:1398] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.835026 28767 consensus_queue.cc:1035] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.836556 29551 consensus_queue.cc:1035] T d89add66b30d49348552022b8f5f8ce3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.842481 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8e4e402a9cb440a5af09d52a47056a0c" } modify_external_catalogs: true
I20240503 02:09:16.842538 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8e4e402a9cb440a5af09d52a47056a0c" } modify_external_catalogs: true
I20240503 02:09:16.843248 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_137a9798ebb04facb2c5307cec82768f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.843374 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_137a9798ebb04facb2c5307cec82768f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.843386 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 322fe14cc0844c2b842ba27e21451535
I20240503 02:09:16.843657 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe14cc0844c2b842ba27e21451535 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.843734 28654 tablet_replica.cc:330] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.843767 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe14cc0844c2b842ba27e21451535 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.843791 28654 raft_consensus.cc:2229] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.843792 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 322fe14cc0844c2b842ba27e21451535 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.843823 28661 tablet_replica.cc:330] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.843823 28654 raft_consensus.cc:2258] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.843860 28653 tablet_replica.cc:330] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.843868 28661 raft_consensus.cc:2229] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.843907 28653 raft_consensus.cc:2229] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.843922 28661 raft_consensus.cc:2258] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.843940 28653 raft_consensus.cc:2258] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.844098 28653 ts_tablet_manager.cc:1875] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.844101 28661 ts_tablet_manager.cc:1875] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.844156 28654 ts_tablet_manager.cc:1875] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.845253 28653 ts_tablet_manager.cc:1888] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.845264 28661 ts_tablet_manager.cc:1888] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.845264 28654 ts_tablet_manager.cc:1888] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.845288 28653 log.cc:1196] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/322fe14cc0844c2b842ba27e21451535
I20240503 02:09:16.845311 28661 log.cc:1196] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/322fe14cc0844c2b842ba27e21451535
I20240503 02:09:16.845311 28654 log.cc:1196] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/322fe14cc0844c2b842ba27e21451535
I20240503 02:09:16.845543 28654 ts_tablet_manager.cc:1909] T 322fe14cc0844c2b842ba27e21451535 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.845543 28661 ts_tablet_manager.cc:1909] T 322fe14cc0844c2b842ba27e21451535 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.845577 28653 ts_tablet_manager.cc:1909] T 322fe14cc0844c2b842ba27e21451535 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.845775 29551 raft_consensus.cc:482] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.845826 29551 raft_consensus.cc:504] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.845930 29551 leader_election.cc:288] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.846074 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 322fe14cc0844c2b842ba27e21451535 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.846184 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 322fe14cc0844c2b842ba27e21451535 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.846197 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9181c05524c3474282c7f526e1195d29" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.846266 27722 raft_consensus.cc:2454] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.846390 27388 leader_election.cc:302] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.846385 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9181c05524c3474282c7f526e1195d29" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.846444 27583 raft_consensus.cc:2454] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.846457 29551 raft_consensus.cc:2781] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.846480 29551 raft_consensus.cc:482] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.846494 29551 raft_consensus.cc:3037] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.846493 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 322fe14cc0844c2b842ba27e21451535 (table default.Table_8e4e402a9cb440a5af09d52a47056a0c [id=162fe7b27a5d4190a92e2b2a11f29b1b]) successfully deleted
I20240503 02:09:16.846987 29551 raft_consensus.cc:504] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.847081 29551 leader_election.cc:288] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.847200 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9181c05524c3474282c7f526e1195d29" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.847259 27722 raft_consensus.cc:3037] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.847847 27692 tablet_service.cc:1467] Processing CreateTablet for tablet a892df6180d248d0b799c34596f382d9 (DEFAULT_TABLE table=default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.847900 27722 raft_consensus.cc:2454] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.847942 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a892df6180d248d0b799c34596f382d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.847960 27425 tablet_service.cc:1467] Processing CreateTablet for tablet a892df6180d248d0b799c34596f382d9 (DEFAULT_TABLE table=default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.847990 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9181c05524c3474282c7f526e1195d29" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.848037 27583 raft_consensus.cc:3037] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.848045 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a892df6180d248d0b799c34596f382d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.848584 27583 raft_consensus.cc:2454] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.848754 27386 leader_election.cc:302] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.848793 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a892df6180d248d0b799c34596f382d9 (DEFAULT_TABLE table=default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.848825 29551 raft_consensus.cc:2781] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.848863 29551 raft_consensus.cc:686] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.848874 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a892df6180d248d0b799c34596f382d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.848899 29551 consensus_queue.cc:237] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.849073 28664 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.849292 28655 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.849436 28664 tablet_bootstrap.cc:654] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.849665 28655 tablet_bootstrap.cc:654] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.849851 28664 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.849802 27202 catalog_manager.cc:5484] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.849895 28664 ts_tablet_manager.cc:1367] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.850071 28656 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.850124 28655 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.850104 28664 raft_consensus.cc:348] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.850153 28664 raft_consensus.cc:374] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.850162 28655 ts_tablet_manager.cc:1367] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.850173 28664 raft_consensus.cc:729] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.850203 28664 consensus_queue.cc:260] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.850299 28664 ts_tablet_manager.cc:1398] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:16.850307 28656 tablet_bootstrap.cc:654] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.850282 28655 raft_consensus.cc:348] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.850351 28655 raft_consensus.cc:374] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.850371 28655 raft_consensus.cc:729] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.850414 28655 consensus_queue.cc:260] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.850495 28655 ts_tablet_manager.cc:1398] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.850832 28656 tablet_bootstrap.cc:492] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.850870 28656 ts_tablet_manager.cc:1367] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.850947 28656 raft_consensus.cc:348] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.850988 28656 raft_consensus.cc:374] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.851003 28656 raft_consensus.cc:729] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.851037 28656 consensus_queue.cc:260] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.851110 28656 ts_tablet_manager.cc:1398] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.852592 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b8c8954eca3b491181c678ac0c417dff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.852689 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8c8954eca3b491181c678ac0c417dff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.852818 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8490179685754442b955ed419b307ac6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.852881 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8490179685754442b955ed419b307ac6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.854773 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 3297f72c695944debdc8c733280bb86a (DEFAULT_TABLE table=default.Table_b8c8954eca3b491181c678ac0c417dff [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.854869 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3297f72c695944debdc8c733280bb86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.854990 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 3297f72c695944debdc8c733280bb86a (DEFAULT_TABLE table=default.Table_b8c8954eca3b491181c678ac0c417dff [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.855036 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 3297f72c695944debdc8c733280bb86a (DEFAULT_TABLE table=default.Table_b8c8954eca3b491181c678ac0c417dff [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.855085 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3297f72c695944debdc8c733280bb86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.855120 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3297f72c695944debdc8c733280bb86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.855926 29550 raft_consensus.cc:482] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.855971 29550 raft_consensus.cc:504] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.855978 28664 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.856077 29550 leader_election.cc:288] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.856155 28655 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.856278 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91b8d143d5c49c19d4a328e714b572d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.856333 28664 tablet_bootstrap.cc:654] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.856335 27583 raft_consensus.cc:2454] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.856335 28656 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.856474 28655 tablet_bootstrap.cc:654] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.856668 28656 tablet_bootstrap.cc:654] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.856683 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91b8d143d5c49c19d4a328e714b572d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.856736 27722 raft_consensus.cc:2454] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.856879 27388 leader_election.cc:302] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.856945 29550 raft_consensus.cc:2781] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.856973 29550 raft_consensus.cc:482] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.856992 29550 raft_consensus.cc:3037] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.857053 28655 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.857088 28655 ts_tablet_manager.cc:1367] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.857149 28664 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.857184 28664 ts_tablet_manager.cc:1367] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.857218 28655 raft_consensus.cc:348] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.857255 28655 raft_consensus.cc:374] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.857265 28655 raft_consensus.cc:729] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.857291 28664 raft_consensus.cc:348] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.857307 28655 consensus_queue.cc:260] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.857375 28664 raft_consensus.cc:374] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.857393 28664 raft_consensus.cc:729] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.857439 28655 ts_tablet_manager.cc:1398] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.857436 28664 consensus_queue.cc:260] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.857520 28664 ts_tablet_manager.cc:1398] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.857614 29550 raft_consensus.cc:504] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.857690 29550 leader_election.cc:288] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.857877 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91b8d143d5c49c19d4a328e714b572d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.857951 27722 raft_consensus.cc:3037] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.858520 27722 raft_consensus.cc:2454] T e91b8d143d5c49c19d4a328e714b572d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.858667 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91b8d143d5c49c19d4a328e714b572d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.858685 27388 leader_election.cc:302] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.858726 27583 raft_consensus.cc:3037] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.858757 29551 raft_consensus.cc:2781] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.858755 27565 tablet_service.cc:1467] Processing CreateTablet for tablet ffac3388531d43d19347097204a8a3d0 (DEFAULT_TABLE table=default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.858791 29551 raft_consensus.cc:686] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.858824 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffac3388531d43d19347097204a8a3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.858829 29551 consensus_queue.cc:237] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.859280 27583 raft_consensus.cc:2454] T e91b8d143d5c49c19d4a328e714b572d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.859876 27692 tablet_service.cc:1467] Processing CreateTablet for tablet ffac3388531d43d19347097204a8a3d0 (DEFAULT_TABLE table=default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.859967 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffac3388531d43d19347097204a8a3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.860056 27425 tablet_service.cc:1467] Processing CreateTablet for tablet ffac3388531d43d19347097204a8a3d0 (DEFAULT_TABLE table=default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.860141 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffac3388531d43d19347097204a8a3d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.860656 27853 consensus_queue.cc:1035] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.860926 28664 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.861145 28655 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.861302 28664 tablet_bootstrap.cc:654] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.861382 29550 raft_consensus.cc:482] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.861433 29550 raft_consensus.cc:504] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.861444 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5706cd992b4448de941e51cf8f3a22ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.861526 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5706cd992b4448de941e51cf8f3a22ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.861523 29550 leader_election.cc:288] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.861701 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949b2d82e23340d3b4ce56934a4db493" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.861766 27588 raft_consensus.cc:2454] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.861825 27853 raft_consensus.cc:482] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.861851 28655 tablet_bootstrap.cc:654] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.861864 27853 raft_consensus.cc:504] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.861966 27853 leader_election.cc:288] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.862131 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a892df6180d248d0b799c34596f382d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.862190 27449 raft_consensus.cc:2454] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.862215 27386 leader_election.cc:302] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.862241 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a892df6180d248d0b799c34596f382d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.862287 27588 raft_consensus.cc:2454] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.862314 29550 raft_consensus.cc:2781] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.862341 29550 raft_consensus.cc:482] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.862357 29550 raft_consensus.cc:3037] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.862887 29550 raft_consensus.cc:504] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.862989 29550 leader_election.cc:288] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.863049 28664 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.863090 28664 ts_tablet_manager.cc:1367] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:16.863145 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949b2d82e23340d3b4ce56934a4db493" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.863204 27718 raft_consensus.cc:3037] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.863215 28664 raft_consensus.cc:348] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.863266 28664 raft_consensus.cc:374] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.863214 27202 catalog_manager.cc:5484] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.863283 28664 raft_consensus.cc:729] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.863315 28664 consensus_queue.cc:260] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.863399 28664 ts_tablet_manager.cc:1398] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.863692 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949b2d82e23340d3b4ce56934a4db493" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.863830 27718 raft_consensus.cc:2454] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.864005 27388 leader_election.cc:302] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.864068 28656 tablet_bootstrap.cc:492] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.864102 28656 ts_tablet_manager.cc:1367] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.864178 28656 raft_consensus.cc:348] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.864223 28656 raft_consensus.cc:374] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.864241 28656 raft_consensus.cc:729] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.864270 28656 consensus_queue.cc:260] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.864349 28656 ts_tablet_manager.cc:1398] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.864375 27659 leader_election.cc:302] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.864406 28656 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.864447 27853 raft_consensus.cc:2781] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.864472 27853 raft_consensus.cc:482] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.864491 27853 raft_consensus.cc:3037] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.864609 28656 tablet_bootstrap.cc:654] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.865024 28656 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.865060 28656 ts_tablet_manager.cc:1367] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.865079 27853 raft_consensus.cc:504] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.865152 28656 raft_consensus.cc:348] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.865195 28656 raft_consensus.cc:374] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.865195 27853 leader_election.cc:288] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.865212 28656 raft_consensus.cc:729] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.865247 28656 consensus_queue.cc:260] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.865342 28656 ts_tablet_manager.cc:1398] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.865491 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a892df6180d248d0b799c34596f382d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.865545 27449 raft_consensus.cc:3037] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.865739 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a892df6180d248d0b799c34596f382d9" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.865796 27583 raft_consensus.cc:3037] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.866150 27449 raft_consensus.cc:2454] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.866261 29550 raft_consensus.cc:2781] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.866297 29550 raft_consensus.cc:686] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.866349 27583 raft_consensus.cc:2454] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.866345 29550 consensus_queue.cc:237] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.865795 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "949b2d82e23340d3b4ce56934a4db493" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.866694 27588 raft_consensus.cc:3037] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.866861 27659 leader_election.cc:302] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.867077 28740 raft_consensus.cc:2781] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.867120 28740 raft_consensus.cc:686] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.867156 28740 consensus_queue.cc:237] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.867226 27588 raft_consensus.cc:2454] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.867332 28740 consensus_queue.cc:1035] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.868746 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b91be7e058484cdeb0b5f8c0bb4bc3b7" } new_table_name: "default.Table_48c8a8213fc54f158347e24c2f4bfe0f" modify_external_catalogs: true
I20240503 02:09:16.868873 28655 tablet_bootstrap.cc:492] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.868908 28655 ts_tablet_manager.cc:1367] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:16.869082 28655 raft_consensus.cc:348] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.869145 28655 raft_consensus.cc:374] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.869163 28655 raft_consensus.cc:729] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.869201 28655 consensus_queue.cc:260] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.869287 28655 ts_tablet_manager.cc:1398] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.869452 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9f8a89570cd247eabbcde7bb9838a07e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.869545 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f8a89570cd247eabbcde7bb9838a07e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.869774 27565 tablet_service.cc:1467] Processing CreateTablet for tablet cab656d127ee405bb11ad94df6eba430 (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.869863 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab656d127ee405bb11ad94df6eba430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.870525 28656 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.870699 27198 catalog_manager.cc:5484] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.871091 28656 tablet_bootstrap.cc:654] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.871174 27197 catalog_manager.cc:5484] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.871630 28656 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.871618 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_666cd538e690425ba8746112d159a266"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.871673 28656 ts_tablet_manager.cc:1367] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
W20240503 02:09:16.871706 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_666cd538e690425ba8746112d159a266 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.871779 28656 raft_consensus.cc:348] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.871825 28656 raft_consensus.cc:374] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.871842 28656 raft_consensus.cc:729] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.871876 28656 consensus_queue.cc:260] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.871961 28656 ts_tablet_manager.cc:1398] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.871970 27425 tablet_service.cc:1467] Processing CreateTablet for tablet cab656d127ee405bb11ad94df6eba430 (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.872052 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab656d127ee405bb11ad94df6eba430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.872190 29550 raft_consensus.cc:482] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.872241 29550 raft_consensus.cc:504] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.872327 29550 leader_election.cc:288] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.872434 29643 raft_consensus.cc:482] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.872479 29643 raft_consensus.cc:504] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.872606 29643 leader_election.cc:288] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.872709 27692 tablet_service.cc:1467] Processing CreateTablet for tablet cab656d127ee405bb11ad94df6eba430 (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.872720 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e0f31cbd417c8cd13696c24f980c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.872769 27582 raft_consensus.cc:2454] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.872799 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab656d127ee405bb11ad94df6eba430. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.872895 27656 leader_election.cc:302] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.872937 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e0f31cbd417c8cd13696c24f980c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.872987 27449 raft_consensus.cc:2454] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.872997 28693 raft_consensus.cc:2781] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.873024 28693 raft_consensus.cc:482] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.873041 28693 raft_consensus.cc:3037] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.873171 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3de7994307949ed9eee04c19f9080e3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.873224 27588 raft_consensus.cc:2454] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.873667 28693 raft_consensus.cc:504] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.873765 28693 leader_election.cc:288] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.873826 28655 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.873867 28664 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.874147 28655 tablet_bootstrap.cc:654] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.874207 28664 tablet_bootstrap.cc:654] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.874440 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3de7994307949ed9eee04c19f9080e3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.874497 27718 raft_consensus.cc:2454] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.874998 27388 leader_election.cc:302] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.875166 29550 raft_consensus.cc:2781] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.875195 29550 raft_consensus.cc:482] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.875211 29550 raft_consensus.cc:3037] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.875330 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e0f31cbd417c8cd13696c24f980c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.875378 27582 raft_consensus.cc:3037] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.875744 29550 raft_consensus.cc:504] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.875840 29550 leader_election.cc:288] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.876037 27582 raft_consensus.cc:2454] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.876195 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1388e0f31cbd417c8cd13696c24f980c" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.876209 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3de7994307949ed9eee04c19f9080e3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.876252 27582 raft_consensus.cc:3037] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.876253 27449 raft_consensus.cc:3037] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.876258 27656 leader_election.cc:302] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.876369 28693 raft_consensus.cc:2781] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.876423 28693 raft_consensus.cc:686] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.876474 28693 consensus_queue.cc:237] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.876770 27582 raft_consensus.cc:2454] T d3de7994307949ed9eee04c19f9080e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.876786 27449 raft_consensus.cc:2454] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.876894 27386 leader_election.cc:302] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.877854 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3de7994307949ed9eee04c19f9080e3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.877915 27718 raft_consensus.cc:3037] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.878244 29550 raft_consensus.cc:2781] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.878295 29550 raft_consensus.cc:686] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.878332 29550 consensus_queue.cc:237] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.878412 27718 raft_consensus.cc:2454] T d3de7994307949ed9eee04c19f9080e3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.878978 27582 raft_consensus.cc:1261] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.878983 27449 raft_consensus.cc:1261] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.879146 29643 consensus_queue.cc:1035] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.879226 28693 consensus_queue.cc:1035] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.879979 28655 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.880021 28655 ts_tablet_manager.cc:1367] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:16.880167 28655 raft_consensus.cc:348] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.880213 28655 raft_consensus.cc:374] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.880228 28655 raft_consensus.cc:729] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.880266 28655 consensus_queue.cc:260] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.880342 28655 ts_tablet_manager.cc:1398] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.880666 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_593fb0f3a09f409f983f5dd9cffd5355"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.880770 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_593fb0f3a09f409f983f5dd9cffd5355 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.880898 29558 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.881278 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_229b7e8c38f64c879016be88401b4602"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:16.881421 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_229b7e8c38f64c879016be88401b4602 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.881826 28710 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.882004 27692 tablet_service.cc:1467] Processing CreateTablet for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (DEFAULT_TABLE table=default.Table_666cd538e690425ba8746112d159a266 [id=5c0a18e04e06431e8311413c9db7a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.881986 27195 catalog_manager.cc:5484] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.882076 28664 tablet_bootstrap.cc:492] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.882117 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.882115 28664 ts_tablet_manager.cc:1367] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:16.882086 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcc99e566bc24ad6a98d20dc61bcfa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.882236 28410 raft_consensus.cc:482] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.882282 28664 raft_consensus.cc:348] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.882282 28410 raft_consensus.cc:504] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.882333 28664 raft_consensus.cc:374] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.882349 28664 raft_consensus.cc:729] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.882371 28664 consensus_queue.cc:260] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.882401 28410 leader_election.cc:288] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.882458 28664 ts_tablet_manager.cc:1398] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.882624 27565 tablet_service.cc:1467] Processing CreateTablet for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (DEFAULT_TABLE table=default.Table_666cd538e690425ba8746112d159a266 [id=5c0a18e04e06431e8311413c9db7a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.882704 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcc99e566bc24ad6a98d20dc61bcfa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.883309 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.883386 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e29e26955d441ef9f7ca908da29ca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.883649 27425 tablet_service.cc:1467] Processing CreateTablet for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (DEFAULT_TABLE table=default.Table_666cd538e690425ba8746112d159a266 [id=5c0a18e04e06431e8311413c9db7a17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.883730 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6873697085ec4322a5871dd591dd81dd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.883792 27449 raft_consensus.cc:2454] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.883843 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.884001 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e29e26955d441ef9f7ca908da29ca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.884189 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6873697085ec4322a5871dd591dd81dd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.884256 27718 raft_consensus.cc:2454] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.884522 28656 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.884531 27523 leader_election.cc:302] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.884693 28410 raft_consensus.cc:2781] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.884717 28410 raft_consensus.cc:482] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.884734 28410 raft_consensus.cc:3037] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.884858 28656 tablet_bootstrap.cc:654] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.883735 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcc99e566bc24ad6a98d20dc61bcfa6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.885377 28410 raft_consensus.cc:504] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.885475 28410 leader_election.cc:288] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.885440 27202 catalog_manager.cc:5484] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.885525 28664 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.885598 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e29e26955d441ef9f7ca908da29ca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.885851 28664 tablet_bootstrap.cc:654] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.886386 28664 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.886428 28664 ts_tablet_manager.cc:1367] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.886422 28655 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.886526 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6873697085ec4322a5871dd591dd81dd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.886546 28664 raft_consensus.cc:348] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.886574 27449 raft_consensus.cc:3037] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.886593 28664 raft_consensus.cc:374] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.886610 28664 raft_consensus.cc:729] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.886642 28664 consensus_queue.cc:260] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.886713 28664 ts_tablet_manager.cc:1398] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.886765 28664 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.886799 28655 tablet_bootstrap.cc:654] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.887089 28664 tablet_bootstrap.cc:654] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.887120 27449 raft_consensus.cc:2454] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.887184 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6873697085ec4322a5871dd591dd81dd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.887244 27718 raft_consensus.cc:3037] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.887575 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b2424d6c3f964a17b69baea107c5cde3" } modify_external_catalogs: true
I20240503 02:09:16.887631 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b2424d6c3f964a17b69baea107c5cde3" } modify_external_catalogs: true
I20240503 02:09:16.887810 27718 raft_consensus.cc:2454] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.887952 27524 leader_election.cc:302] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.888020 28410 raft_consensus.cc:2781] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.888051 28410 raft_consensus.cc:686] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.888085 28410 consensus_queue.cc:237] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.889614 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_87eb7af059c842d6b7b4c8e2be015384" } new_table_name: "default.Table_a81c565ca83042d6bd00fc3c7cb4eb3c" modify_external_catalogs: true
I20240503 02:09:16.889850 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 82e659f5818744f584e64997789ebf86 (table default.Table_48c8a8213fc54f158347e24c2f4bfe0f [id=c10aff1a98f64f13a7ffb57bf8a6e815]). Expected version 1 got 0
I20240503 02:09:16.890007 28655 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.890043 28655 ts_tablet_manager.cc:1367] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:16.890162 28655 raft_consensus.cc:348] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.890213 28655 raft_consensus.cc:374] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.890229 28655 raft_consensus.cc:729] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.890237 28656 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.890261 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1186037b246e4c70ac3196564d599c54
I20240503 02:09:16.890275 28656 ts_tablet_manager.cc:1367] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:16.890265 28655 consensus_queue.cc:260] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.890350 28655 ts_tablet_manager.cc:1398] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.890410 28655 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.890430 28656 raft_consensus.cc:348] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.890486 28656 raft_consensus.cc:374] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.890501 28656 raft_consensus.cc:729] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.890535 28656 consensus_queue.cc:260] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.890547 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 1186037b246e4c70ac3196564d599c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.890620 28654 tablet_replica.cc:330] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.890635 28656 ts_tablet_manager.cc:1398] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.890663 28654 raft_consensus.cc:2229] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.890687 28656 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.890741 28655 tablet_bootstrap.cc:654] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.890949 28664 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.890998 28664 ts_tablet_manager.cc:1367] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:16.891146 28664 raft_consensus.cc:348] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.891202 28664 raft_consensus.cc:374] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.891220 28664 raft_consensus.cc:729] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.891255 28664 consensus_queue.cc:260] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.891290 28655 tablet_bootstrap.cc:492] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.891328 28655 ts_tablet_manager.cc:1367] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.891332 28664 ts_tablet_manager.cc:1398] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.891444 28655 raft_consensus.cc:348] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.890691 28654 raft_consensus.cc:2258] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.891495 28655 raft_consensus.cc:374] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.891513 28655 raft_consensus.cc:729] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.891537 28655 consensus_queue.cc:260] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.891614 28655 ts_tablet_manager.cc:1398] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.891649 28654 ts_tablet_manager.cc:1875] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.892406 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 1186037b246e4c70ac3196564d599c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.892590 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 2b675ecf68b049edac996e4fc7b0bbd8 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.892971 28654 ts_tablet_manager.cc:1888] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.893018 28654 log.cc:1196] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1186037b246e4c70ac3196564d599c54
I20240503 02:09:16.893160 28654 ts_tablet_manager.cc:1909] T 1186037b246e4c70ac3196564d599c54 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.893271 28661 tablet_replica.cc:330] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.893373 28661 raft_consensus.cc:2229] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.893435 28661 pending_rounds.cc:62] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:16.893450 28661 pending_rounds.cc:68] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420033962926080 op_type: NO_OP noop_request { }
I20240503 02:09:16.893476 28661 raft_consensus.cc:2866] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.893494 28661 raft_consensus.cc:2258] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.893976 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 1186037b246e4c70ac3196564d599c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.894111 28653 tablet_replica.cc:330] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.894181 28653 raft_consensus.cc:2229] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.894198 28661 ts_tablet_manager.cc:1875] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.894217 28653 raft_consensus.cc:2258] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.894445 28653 ts_tablet_manager.cc:1875] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.895458 28661 ts_tablet_manager.cc:1888] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.895500 28661 log.cc:1196] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1186037b246e4c70ac3196564d599c54
I20240503 02:09:16.895689 28661 ts_tablet_manager.cc:1909] T 1186037b246e4c70ac3196564d599c54 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.895720 28653 ts_tablet_manager.cc:1888] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.895756 28653 log.cc:1196] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1186037b246e4c70ac3196564d599c54
I20240503 02:09:16.895896 28653 ts_tablet_manager.cc:1909] T 1186037b246e4c70ac3196564d599c54 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.896270 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_445a2ff5353846579f6884b81b92e97a" } new_table_name: "default.Table_59f76355e5b649b68043222a5d0cd38a" modify_external_catalogs: true
I20240503 02:09:16.896554 28656 tablet_bootstrap.cc:654] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.897019 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_663f85401b7743828f1c24e63c7f40ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:16.897037 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1186037b246e4c70ac3196564d599c54 (table default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]) successfully deleted
W20240503 02:09:16.897131 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_663f85401b7743828f1c24e63c7f40ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:16.897397 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1186037b246e4c70ac3196564d599c54 (table default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]) successfully deleted
I20240503 02:09:16.897565 28656 tablet_bootstrap.cc:492] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.897605 28656 ts_tablet_manager.cc:1367] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:16.897715 28656 raft_consensus.cc:348] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.897768 28656 raft_consensus.cc:374] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.897785 28656 raft_consensus.cc:729] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.897817 28656 consensus_queue.cc:260] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.897917 28656 ts_tablet_manager.cc:1398] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.898662 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1186037b246e4c70ac3196564d599c54 (table default.Table_b2424d6c3f964a17b69baea107c5cde3 [id=64c59a62a41a4bdebd98d424685d1133]) successfully deleted
I20240503 02:09:16.898895 29558 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.899816 28710 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.899976 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2b675ecf68b049edac996e4fc7b0bbd8
I20240503 02:09:16.900038 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 2b675ecf68b049edac996e4fc7b0bbd8 deleted and replaced by tablet a3650180caf546878779d1c89950b2c1
I20240503 02:09:16.900216 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3650180caf546878779d1c89950b2c1) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.900265 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_48c8a8213fc54f158347e24c2f4bfe0f [id=c10aff1a98f64f13a7ffb57bf8a6e815] alter complete (version 1)
I20240503 02:09:16.900293 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a7116932a247407196f5c17ddc1b4e9f (DEFAULT_TABLE table=default.Table_593fb0f3a09f409f983f5dd9cffd5355 [id=a38d6ee085a74d4eb03faa3f91cf5610]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.900362 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7116932a247407196f5c17ddc1b4e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.900377 28661 tablet_replica.cc:330] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.900436 28661 raft_consensus.cc:2229] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.900468 28661 raft_consensus.cc:2258] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.900571 28661 ts_tablet_manager.cc:1875] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.901517 27427 tablet_service.cc:1467] Processing CreateTablet for tablet a7116932a247407196f5c17ddc1b4e9f (DEFAULT_TABLE table=default.Table_593fb0f3a09f409f983f5dd9cffd5355 [id=a38d6ee085a74d4eb03faa3f91cf5610]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.901600 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7116932a247407196f5c17ddc1b4e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.901646 28661 ts_tablet_manager.cc:1888] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.901690 28661 log.cc:1196] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2b675ecf68b049edac996e4fc7b0bbd8
I20240503 02:09:16.901857 28661 ts_tablet_manager.cc:1909] T 2b675ecf68b049edac996e4fc7b0bbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.902024 27201 catalog_manager.cc:5484] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.902122 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.902643 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3650180caf546878779d1c89950b2c1) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.902645 28655 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.902931 28653 tablet_replica.cc:330] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.902981 28653 raft_consensus.cc:2229] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.903012 28653 raft_consensus.cc:2258] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.903108 28653 ts_tablet_manager.cc:1875] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.903117 28655 tablet_bootstrap.cc:654] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.903657 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ddd8e5eab2b84d9da38e58ee1e2e436e" } new_table_name: "default.Table_24404d0b2efa43feb938c7ffd004e375" modify_external_catalogs: true
I20240503 02:09:16.903733 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a7116932a247407196f5c17ddc1b4e9f (DEFAULT_TABLE table=default.Table_593fb0f3a09f409f983f5dd9cffd5355 [id=a38d6ee085a74d4eb03faa3f91cf5610]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.903816 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7116932a247407196f5c17ddc1b4e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.904227 28653 ts_tablet_manager.cc:1888] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.904278 28653 log.cc:1196] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2b675ecf68b049edac996e4fc7b0bbd8
I20240503 02:09:16.904446 28653 ts_tablet_manager.cc:1909] T 2b675ecf68b049edac996e4fc7b0bbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.904997 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b675ecf68b049edac996e4fc7b0bbd8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3650180caf546878779d1c89950b2c1) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.905066 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2b675ecf68b049edac996e4fc7b0bbd8 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:16.905661 28656 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.906108 28656 tablet_bootstrap.cc:654] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.906508 28654 tablet_replica.cc:330] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.906558 28664 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.906579 28654 raft_consensus.cc:2229] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.906636 28654 raft_consensus.cc:2258] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.906734 28654 ts_tablet_manager.cc:1875] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.906896 28664 tablet_bootstrap.cc:654] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.907341 28664 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.907379 28664 ts_tablet_manager.cc:1367] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.907455 28664 raft_consensus.cc:348] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.907514 28664 raft_consensus.cc:374] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.907547 28664 raft_consensus.cc:729] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.907588 28664 consensus_queue.cc:260] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.907675 28664 ts_tablet_manager.cc:1398] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.907768 28654 ts_tablet_manager.cc:1888] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.907809 28654 log.cc:1196] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2b675ecf68b049edac996e4fc7b0bbd8
I20240503 02:09:16.907975 28654 ts_tablet_manager.cc:1909] T 2b675ecf68b049edac996e4fc7b0bbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.908706 29558 tablet.cc:1722] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.908993 28710 tablet.cc:1722] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.909570 29566 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.909790 29167 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.910727 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2b675ecf68b049edac996e4fc7b0bbd8 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:16.911211 28698 tablet.cc:1722] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.911469 29558 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.911573 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 1388e0f31cbd417c8cd13696c24f980c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.911592 29566 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.911854 28655 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.911890 28655 ts_tablet_manager.cc:1367] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:16.912011 28655 raft_consensus.cc:348] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.912062 28655 raft_consensus.cc:374] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.912081 28655 raft_consensus.cc:729] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.912119 28655 consensus_queue.cc:260] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.912207 28655 ts_tablet_manager.cc:1398] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.912503 28656 tablet_bootstrap.cc:492] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.912561 28656 ts_tablet_manager.cc:1367] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:16.912711 28656 raft_consensus.cc:348] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.912768 28656 raft_consensus.cc:374] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.912784 28656 raft_consensus.cc:729] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.912811 28656 consensus_queue.cc:260] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.912899 28656 ts_tablet_manager.cc:1398] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.913667 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2b675ecf68b049edac996e4fc7b0bbd8 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:16.914285 28710 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:16.915357 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 69ea8592a1154e4e80f75853e2664e07 (DEFAULT_TABLE table=default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.915448 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ea8592a1154e4e80f75853e2664e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.915725 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 0a549a5309d0451a8b58607ecdc3378a (DEFAULT_TABLE table=default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.915742 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 0a549a5309d0451a8b58607ecdc3378a (DEFAULT_TABLE table=default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.915803 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a549a5309d0451a8b58607ecdc3378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.915824 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a549a5309d0451a8b58607ecdc3378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.916467 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a3650180caf546878779d1c89950b2c1 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.916548 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3650180caf546878779d1c89950b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.916539 28655 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.916895 28655 tablet_bootstrap.cc:654] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.917084 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 69ea8592a1154e4e80f75853e2664e07 (DEFAULT_TABLE table=default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.917162 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ea8592a1154e4e80f75853e2664e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.917155 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1388e0f31cbd417c8cd13696c24f980c
I20240503 02:09:16.917230 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 1388e0f31cbd417c8cd13696c24f980c deleted and replaced by tablet 5e1800d45b4f4593aa21127c2fc93f56
I20240503 02:09:16.917436 27200 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_a81c565ca83042d6bd00fc3c7cb4eb3c [id=5ff15871699b4869b63a0187224a6cd7] alter complete (version 2)
I20240503 02:09:16.917754 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 69ea8592a1154e4e80f75853e2664e07 (DEFAULT_TABLE table=default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.917836 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ea8592a1154e4e80f75853e2664e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.918192 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e0f31cbd417c8cd13696c24f980c with delete_type TABLET_DATA_DELETED (replaced by tablet 5e1800d45b4f4593aa21127c2fc93f56) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.918262 28653 tablet_replica.cc:330] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.918310 28653 raft_consensus.cc:2229] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.918341 28653 raft_consensus.cc:2258] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.918326 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a3650180caf546878779d1c89950b2c1 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.918401 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3650180caf546878779d1c89950b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.918519 28653 ts_tablet_manager.cc:1875] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.918947 27427 tablet_service.cc:1467] Processing CreateTablet for tablet a3650180caf546878779d1c89950b2c1 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.919018 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3650180caf546878779d1c89950b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.919400 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 0a549a5309d0451a8b58607ecdc3378a (DEFAULT_TABLE table=default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.919476 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a549a5309d0451a8b58607ecdc3378a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.919783 28653 ts_tablet_manager.cc:1888] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.919816 28653 log.cc:1196] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1388e0f31cbd417c8cd13696c24f980c
I20240503 02:09:16.919916 28653 ts_tablet_manager.cc:1909] T 1388e0f31cbd417c8cd13696c24f980c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.920238 28656 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.920588 28656 tablet_bootstrap.cc:654] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.920737 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a273104fb204435d9b452271f100bb80" } new_table_name: "default.Table_dfd597292ebc4b99a89f252844934cc6" modify_external_catalogs: true
I20240503 02:09:16.920768 28664 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.921028 28664 tablet_bootstrap.cc:654] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.921190 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1b09b97f30ac43ecbbeac65f8366738a" } modify_external_catalogs: true
I20240503 02:09:16.921252 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1b09b97f30ac43ecbbeac65f8366738a" } modify_external_catalogs: true
I20240503 02:09:16.921305 28656 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.921402 28656 ts_tablet_manager.cc:1367] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.921408 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e0f31cbd417c8cd13696c24f980c with delete_type TABLET_DATA_DELETED (replaced by tablet 5e1800d45b4f4593aa21127c2fc93f56) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.921479 28661 tablet_replica.cc:330] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.921540 28661 raft_consensus.cc:2229] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.921525 28656 raft_consensus.cc:348] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.921571 28656 raft_consensus.cc:374] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.921588 28661 pending_rounds.cc:62] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:16.921587 28656 raft_consensus.cc:729] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.921607 28661 pending_rounds.cc:68] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420034566324224 op_type: NO_OP noop_request { }
I20240503 02:09:16.921644 28661 raft_consensus.cc:2866] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.921628 28656 consensus_queue.cc:260] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.921663 28661 raft_consensus.cc:2258] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.921710 28656 ts_tablet_manager.cc:1398] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.921590 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1800d45b4f4593aa21127c2fc93f56 (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.921741 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1800d45b4f4593aa21127c2fc93f56 (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.921768 28656 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.921813 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1800d45b4f4593aa21127c2fc93f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.921813 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1800d45b4f4593aa21127c2fc93f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.922041 28661 ts_tablet_manager.cc:1875] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.922083 28656 tablet_bootstrap.cc:654] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.922915 28656 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.922957 28656 ts_tablet_manager.cc:1367] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.922995 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 1388e0f31cbd417c8cd13696c24f980c with delete_type TABLET_DATA_DELETED (replaced by tablet 5e1800d45b4f4593aa21127c2fc93f56) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.923092 28654 tablet_replica.cc:330] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.923135 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1800d45b4f4593aa21127c2fc93f56 (DEFAULT_TABLE table=default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.923084 28656 raft_consensus.cc:348] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.923174 28654 raft_consensus.cc:2229] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.923184 28656 raft_consensus.cc:374] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.923210 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1800d45b4f4593aa21127c2fc93f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.923204 28654 raft_consensus.cc:2258] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.923207 28656 raft_consensus.cc:729] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.923292 28656 consensus_queue.cc:260] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.923359 28661 ts_tablet_manager.cc:1888] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.923406 28661 log.cc:1196] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1388e0f31cbd417c8cd13696c24f980c
I20240503 02:09:16.923398 28656 ts_tablet_manager.cc:1398] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.923470 28656 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.923592 28661 ts_tablet_manager.cc:1909] T 1388e0f31cbd417c8cd13696c24f980c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.923872 28654 ts_tablet_manager.cc:1875] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.923900 28656 tablet_bootstrap.cc:654] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.924906 28656 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.924940 28656 ts_tablet_manager.cc:1367] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.925043 28656 raft_consensus.cc:348] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.925086 28656 raft_consensus.cc:374] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.925103 28656 raft_consensus.cc:729] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.925174 28656 consensus_queue.cc:260] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.925251 28656 ts_tablet_manager.cc:1398] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.925292 28656 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.925280 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1388e0f31cbd417c8cd13696c24f980c (table default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]) successfully deleted
I20240503 02:09:16.925673 28656 tablet_bootstrap.cc:654] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.926033 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1388e0f31cbd417c8cd13696c24f980c (table default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]) successfully deleted
I20240503 02:09:16.925107 28654 ts_tablet_manager.cc:1888] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.926939 28654 log.cc:1196] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1388e0f31cbd417c8cd13696c24f980c
I20240503 02:09:16.927043 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 739e17504abc42d0a54cd92b1f171fca from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.927109 28654 ts_tablet_manager.cc:1909] T 1388e0f31cbd417c8cd13696c24f980c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.927264 27200 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_24404d0b2efa43feb938c7ffd004e375 [id=80cf6485f989436d8fab264d6a5743b4] alter complete (version 2)
I20240503 02:09:16.927440 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1388e0f31cbd417c8cd13696c24f980c (table default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2 [id=24f3eb6c203b452cac076beda486824a]) successfully deleted
I20240503 02:09:16.928432 29558 tablet.cc:1722] T 47a4f79dd7ac4d929abda2d67aad87d9 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.928467 28664 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.928506 28664 ts_tablet_manager.cc:1367] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:16.928630 28664 raft_consensus.cc:348] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.928687 28664 raft_consensus.cc:374] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.928704 28664 raft_consensus.cc:729] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.928740 28664 consensus_queue.cc:260] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.928828 28664 ts_tablet_manager.cc:1398] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.928880 28664 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.929282 28664 tablet_bootstrap.cc:654] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.929762 28664 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.929805 28664 ts_tablet_manager.cc:1367] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.929824 28655 tablet_bootstrap.cc:492] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.929859 28655 ts_tablet_manager.cc:1367] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:16.929858 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_59f76355e5b649b68043222a5d0cd38a [id=da76a80d32d34ee0b39f826d114a51ac] alter complete (version 1)
I20240503 02:09:16.929929 28664 raft_consensus.cc:348] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.930011 28655 raft_consensus.cc:348] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.930063 28655 raft_consensus.cc:374] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.930079 28655 raft_consensus.cc:729] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.930065 29566 tablet.cc:1722] T 47a4f79dd7ac4d929abda2d67aad87d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.930084 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b16f1976661e4dba8b366773533ca59e
I20240503 02:09:16.930116 28655 consensus_queue.cc:260] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.930202 28655 ts_tablet_manager.cc:1398] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.930254 28655 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.930596 28655 tablet_bootstrap.cc:654] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.931073 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f1976661e4dba8b366773533ca59e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.929980 28664 raft_consensus.cc:374] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.931237 28664 raft_consensus.cc:729] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.931299 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f1976661e4dba8b366773533ca59e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.931336 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 739e17504abc42d0a54cd92b1f171fca
I20240503 02:09:16.931370 28654 tablet_replica.cc:330] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.930891 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet b16f1976661e4dba8b366773533ca59e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.931407 27202 catalog_manager.cc:6554] ReplaceTablet: tablet 739e17504abc42d0a54cd92b1f171fca deleted and replaced by tablet 53cd092cf71e47fcad1f2f9091953141
I20240503 02:09:16.931419 28654 raft_consensus.cc:2229] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.931440 28653 tablet_replica.cc:330] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.931452 28654 raft_consensus.cc:2258] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.931501 28653 raft_consensus.cc:2229] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.931541 28653 raft_consensus.cc:2258] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.931599 28661 tablet_replica.cc:330] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.931329 28664 consensus_queue.cc:260] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.931648 28661 raft_consensus.cc:2229] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.931663 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 739e17504abc42d0a54cd92b1f171fca with delete_type TABLET_DATA_DELETED (replaced by tablet 53cd092cf71e47fcad1f2f9091953141) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.931702 28661 raft_consensus.cc:2258] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.931799 28664 ts_tablet_manager.cc:1398] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:16.931856 28664 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.931885 28654 ts_tablet_manager.cc:1875] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.932093 28661 ts_tablet_manager.cc:1875] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.932220 28710 tablet.cc:1722] T 47a4f79dd7ac4d929abda2d67aad87d9 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.932265 28664 tablet_bootstrap.cc:654] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.932763 28664 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.932801 28664 ts_tablet_manager.cc:1367] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
W20240503 02:09:16.932796 27200 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:16.932916 28664 raft_consensus.cc:348] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.932965 28664 raft_consensus.cc:374] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.932982 28664 raft_consensus.cc:729] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.933022 28664 consensus_queue.cc:260] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.933107 28654 ts_tablet_manager.cc:1888] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.933121 28664 ts_tablet_manager.cc:1398] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.933142 28654 log.cc:1196] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b16f1976661e4dba8b366773533ca59e
I20240503 02:09:16.933176 28664 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.933349 28654 ts_tablet_manager.cc:1909] T b16f1976661e4dba8b366773533ca59e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.933395 28661 ts_tablet_manager.cc:1888] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.933462 28661 log.cc:1196] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b16f1976661e4dba8b366773533ca59e
I20240503 02:09:16.933531 28664 tablet_bootstrap.cc:654] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.933643 28661 ts_tablet_manager.cc:1909] T b16f1976661e4dba8b366773533ca59e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.933796 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 739e17504abc42d0a54cd92b1f171fca with delete_type TABLET_DATA_DELETED (replaced by tablet 53cd092cf71e47fcad1f2f9091953141) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.933858 28661 tablet_replica.cc:330] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.933913 28661 raft_consensus.cc:2229] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.933962 28661 raft_consensus.cc:2258] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.934041 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]) successfully deleted
I20240503 02:09:16.934054 28661 ts_tablet_manager.cc:1875] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.934108 28664 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.934144 28664 ts_tablet_manager.cc:1367] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.934293 28664 raft_consensus.cc:348] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.934343 28664 raft_consensus.cc:374] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.934360 28664 raft_consensus.cc:729] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.934396 28664 consensus_queue.cc:260] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.934473 28664 ts_tablet_manager.cc:1398] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.935197 28661 ts_tablet_manager.cc:1888] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.935238 28661 log.cc:1196] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/739e17504abc42d0a54cd92b1f171fca
I20240503 02:09:16.935420 28661 ts_tablet_manager.cc:1909] T 739e17504abc42d0a54cd92b1f171fca P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.935642 28653 ts_tablet_manager.cc:1875] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.935827 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:16.936925 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 739e17504abc42d0a54cd92b1f171fca with delete_type TABLET_DATA_DELETED (replaced by tablet 53cd092cf71e47fcad1f2f9091953141) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.936965 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_443f4d53ffca4f209904f7da04bed3be" } modify_external_catalogs: true
I20240503 02:09:16.937029 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_443f4d53ffca4f209904f7da04bed3be" } modify_external_catalogs: true
I20240503 02:09:16.937224 28656 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.937268 28656 ts_tablet_manager.cc:1367] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:16.937435 28656 raft_consensus.cc:348] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.937489 28656 raft_consensus.cc:374] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.937513 28656 raft_consensus.cc:729] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.937556 28656 consensus_queue.cc:260] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.937640 28656 ts_tablet_manager.cc:1398] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.938688 28653 ts_tablet_manager.cc:1888] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.938738 28653 log.cc:1196] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b16f1976661e4dba8b366773533ca59e
I20240503 02:09:16.938930 28653 ts_tablet_manager.cc:1909] T b16f1976661e4dba8b366773533ca59e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.939118 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 53cd092cf71e47fcad1f2f9091953141 (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.939154 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:16.939221 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cd092cf71e47fcad1f2f9091953141. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.939232 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]) successfully deleted
I20240503 02:09:16.939347 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 53cd092cf71e47fcad1f2f9091953141 (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.939419 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cd092cf71e47fcad1f2f9091953141. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.939126 28653 tablet_replica.cc:330] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.939481 28653 raft_consensus.cc:2229] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.939515 28653 raft_consensus.cc:2258] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.939611 28653 ts_tablet_manager.cc:1875] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.939639 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 010d95e45f6e417c95d4d3e6278f4d74
I20240503 02:09:16.940245 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.940268 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.940425 28664 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.940455 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 53cd092cf71e47fcad1f2f9091953141 (DEFAULT_TABLE table=default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.940532 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53cd092cf71e47fcad1f2f9091953141. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.940666 28664 tablet_bootstrap.cc:654] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.940821 28653 ts_tablet_manager.cc:1888] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.940863 28653 log.cc:1196] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/739e17504abc42d0a54cd92b1f171fca
I20240503 02:09:16.941038 28653 ts_tablet_manager.cc:1909] T 739e17504abc42d0a54cd92b1f171fca P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.941233 28653 tablet_replica.cc:330] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.941290 28664 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.941294 28653 raft_consensus.cc:2229] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.941354 28664 ts_tablet_manager.cc:1367] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.941459 28664 raft_consensus.cc:348] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.941519 28664 raft_consensus.cc:374] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.941541 28664 raft_consensus.cc:729] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.941573 28664 consensus_queue.cc:260] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.941617 28654 tablet_replica.cc:330] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.941654 28664 ts_tablet_manager.cc:1398] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.941668 28654 raft_consensus.cc:2229] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.941701 28654 raft_consensus.cc:2258] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.941782 28656 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.941803 28654 ts_tablet_manager.cc:1875] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.941884 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b16f1976661e4dba8b366773533ca59e (table default.Table_1b09b97f30ac43ecbbeac65f8366738a [id=4b102e807c7a4aab83ecf1d206d8f80d]) successfully deleted
I20240503 02:09:16.941977 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:16.942108 28656 tablet_bootstrap.cc:654] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.942312 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 010d95e45f6e417c95d4d3e6278f4d74 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:16 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.942925 28661 tablet_replica.cc:330] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.942979 28654 ts_tablet_manager.cc:1888] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.943020 28654 log.cc:1196] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/739e17504abc42d0a54cd92b1f171fca
I20240503 02:09:16.943017 28661 raft_consensus.cc:2229] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.943078 28661 raft_consensus.cc:2258] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.943217 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_dfd597292ebc4b99a89f252844934cc6 [id=b201298613b94cd79c63ab8832cde060] alter complete (version 1)
I20240503 02:09:16.943568 28661 ts_tablet_manager.cc:1875] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240503 02:09:16.943645 27200 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet a3650180caf546878779d1c89950b2c1 consensus state has no leader
I20240503 02:09:16.944056 28654 ts_tablet_manager.cc:1909] T 739e17504abc42d0a54cd92b1f171fca P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.944497 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet a99054ca9753450db1ac75a35ae25e55 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.944569 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 739e17504abc42d0a54cd92b1f171fca (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:16.944612 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet fa960c0281414895a12972c4c1e5269c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:16.944741 28653 raft_consensus.cc:2258] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.944849 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:16.944906 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
W20240503 02:09:16.944965 27201 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 53cd092cf71e47fcad1f2f9091953141 consensus state has no leader
I20240503 02:09:16.944460 28654 tablet_replica.cc:330] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.945672 28661 ts_tablet_manager.cc:1888] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.945763 28654 raft_consensus.cc:2229] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.945811 28654 raft_consensus.cc:2258] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.945873 28661 log.cc:1196] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/010d95e45f6e417c95d4d3e6278f4d74
I20240503 02:09:16.945899 28653 ts_tablet_manager.cc:1875] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.945904 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet fa960c0281414895a12972c4c1e5269c
I20240503 02:09:16.945991 27202 catalog_manager.cc:6554] ReplaceTablet: tablet fa960c0281414895a12972c4c1e5269c deleted and replaced by tablet cec4ddee00974610917b834da66647d3
I20240503 02:09:16.946090 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet fa960c0281414895a12972c4c1e5269c with delete_type TABLET_DATA_DELETED (replaced by tablet cec4ddee00974610917b834da66647d3) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.946117 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet fa960c0281414895a12972c4c1e5269c with delete_type TABLET_DATA_DELETED (replaced by tablet cec4ddee00974610917b834da66647d3) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.946239 28661 ts_tablet_manager.cc:1909] T 010d95e45f6e417c95d4d3e6278f4d74 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.946280 29551 raft_consensus.cc:482] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.946318 29551 raft_consensus.cc:504] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.946360 28654 ts_tablet_manager.cc:1875] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.946384 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet fa960c0281414895a12972c4c1e5269c with delete_type TABLET_DATA_DELETED (replaced by tablet cec4ddee00974610917b834da66647d3) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.946446 29551 leader_election.cc:288] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.946506 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a99054ca9753450db1ac75a35ae25e55
I20240503 02:09:16.946518 27718 raft_consensus.cc:1261] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.946597 27200 catalog_manager.cc:6554] ReplaceTablet: tablet a99054ca9753450db1ac75a35ae25e55 deleted and replaced by tablet 2983e9a39aed45f29aaaca9c5c010bb8
I20240503 02:09:16.946628 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b715e519091b49858b528eb83bf33875" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.946646 27848 raft_consensus.cc:482] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.946676 27718 raft_consensus.cc:2454] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.946681 27848 raft_consensus.cc:504] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.946678 28661 tablet_replica.cc:330] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.946765 27848 leader_election.cc:288] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.946789 28661 raft_consensus.cc:2229] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.946753 28507 consensus_queue.cc:1035] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.946861 27388 leader_election.cc:302] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.946857 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 010d95e45f6e417c95d4d3e6278f4d74 (table default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7]) successfully deleted
I20240503 02:09:16.946895 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet a99054ca9753450db1ac75a35ae25e55 with delete_type TABLET_DATA_DELETED (replaced by tablet 2983e9a39aed45f29aaaca9c5c010bb8) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:16.946823 28661 raft_consensus.cc:2258] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.946933 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet a99054ca9753450db1ac75a35ae25e55 with delete_type TABLET_DATA_DELETED (replaced by tablet 2983e9a39aed45f29aaaca9c5c010bb8) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:16.946982 29551 raft_consensus.cc:2781] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.947011 29551 raft_consensus.cc:482] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.947031 29551 raft_consensus.cc:3037] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.947057 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet a99054ca9753450db1ac75a35ae25e55 with delete_type TABLET_DATA_DELETED (replaced by tablet 2983e9a39aed45f29aaaca9c5c010bb8) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:16.947589 29551 raft_consensus.cc:504] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.947664 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa67973d02924925a35b4418e1d4fa29" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.947713 29551 leader_election.cc:288] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.947683 27449 raft_consensus.cc:1261] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:16.947734 27451 raft_consensus.cc:2454] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.947880 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b715e519091b49858b528eb83bf33875" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.947914 27524 leader_election.cc:302] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.947943 27582 raft_consensus.cc:2454] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.948024 27848 consensus_queue.cc:1035] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.948088 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b715e519091b49858b528eb83bf33875" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.948128 27848 raft_consensus.cc:2781] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.948144 27718 raft_consensus.cc:3037] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.948151 27848 raft_consensus.cc:482] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.948168 27848 raft_consensus.cc:3037] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.948691 27718 raft_consensus.cc:2454] T b715e519091b49858b528eb83bf33875 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.948743 27848 raft_consensus.cc:504] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.948774 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa67973d02924925a35b4418e1d4fa29" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.948817 27722 raft_consensus.cc:2454] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.948833 27848 leader_election.cc:288] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.948864 27388 leader_election.cc:302] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.949018 29551 raft_consensus.cc:2781] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.949065 29551 raft_consensus.cc:686] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.949092 28661 ts_tablet_manager.cc:1875] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.949106 29551 consensus_queue.cc:237] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.949240 28653 ts_tablet_manager.cc:1888] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.949280 28653 log.cc:1196] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/010d95e45f6e417c95d4d3e6278f4d74
I20240503 02:09:16.949489 28653 ts_tablet_manager.cc:1909] T 010d95e45f6e417c95d4d3e6278f4d74 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.949524 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa67973d02924925a35b4418e1d4fa29" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.949579 27722 raft_consensus.cc:3037] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.949687 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.949707 28653 tablet_replica.cc:330] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.949762 28653 raft_consensus.cc:2229] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.949836 28653 raft_consensus.cc:2258] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.950044 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b715e519091b49858b528eb83bf33875" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.950101 27582 raft_consensus.cc:3037] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.950106 28653 ts_tablet_manager.cc:1875] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.950194 27722 raft_consensus.cc:2454] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.950340 27523 leader_election.cc:302] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.950364 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa67973d02924925a35b4418e1d4fa29" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.950416 27449 raft_consensus.cc:3037] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.950417 28507 raft_consensus.cc:2781] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.950446 28507 raft_consensus.cc:686] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.950479 28507 consensus_queue.cc:237] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.950680 27582 raft_consensus.cc:2454] T b715e519091b49858b528eb83bf33875 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.951069 27449 raft_consensus.cc:2454] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.951517 28653 ts_tablet_manager.cc:1888] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.951563 28653 log.cc:1196] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fa960c0281414895a12972c4c1e5269c
I20240503 02:09:16.951761 28653 ts_tablet_manager.cc:1909] T fa960c0281414895a12972c4c1e5269c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.951862 28661 ts_tablet_manager.cc:1888] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.951972 28653 tablet_replica.cc:330] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:16.952049 28653 raft_consensus.cc:2229] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.952136 28653 raft_consensus.cc:2258] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.952149 28661 log.cc:1196] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fa960c0281414895a12972c4c1e5269c
I20240503 02:09:16.952351 28661 ts_tablet_manager.cc:1909] T fa960c0281414895a12972c4c1e5269c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.952467 28653 ts_tablet_manager.cc:1875] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.952450 27202 catalog_manager.cc:5484] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.952625 27200 catalog_manager.cc:5484] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.952657 28661 tablet_replica.cc:330] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:16.952702 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.952751 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fa960c0281414895a12972c4c1e5269c (table default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]) successfully deleted
I20240503 02:09:16.952713 28661 raft_consensus.cc:2229] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:16.952822 27702 tablet_service.cc:1467] Processing CreateTablet for tablet cec4ddee00974610917b834da66647d3 (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.952831 28661 pending_rounds.cc:62] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:16.952875 28661 pending_rounds.cc:68] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420034253770752 op_type: NO_OP noop_request { }
I20240503 02:09:16.952116 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 010d95e45f6e417c95d4d3e6278f4d74 (table default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7]) successfully deleted
I20240503 02:09:16.952910 28661 raft_consensus.cc:2866] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:16.952913 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec4ddee00974610917b834da66647d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.952966 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fa960c0281414895a12972c4c1e5269c (table default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]) successfully deleted
I20240503 02:09:16.952958 27562 tablet_service.cc:1467] Processing CreateTablet for tablet cec4ddee00974610917b834da66647d3 (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.952948 28661 raft_consensus.cc:2258] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.953040 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec4ddee00974610917b834da66647d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.953163 27427 tablet_service.cc:1467] Processing CreateTablet for tablet cec4ddee00974610917b834da66647d3 (DEFAULT_TABLE table=default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.953495 28661 ts_tablet_manager.cc:1875] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.954360 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cec4ddee00974610917b834da66647d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.954756 28664 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.955792 28653 ts_tablet_manager.cc:1888] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.955833 28653 log.cc:1196] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a99054ca9753450db1ac75a35ae25e55
I20240503 02:09:16.955989 28653 ts_tablet_manager.cc:1909] T a99054ca9753450db1ac75a35ae25e55 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:16.956267 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 2983e9a39aed45f29aaaca9c5c010bb8 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.956365 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2983e9a39aed45f29aaaca9c5c010bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.956413 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a99054ca9753450db1ac75a35ae25e55 (table default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]) successfully deleted
I20240503 02:09:16.956423 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 2983e9a39aed45f29aaaca9c5c010bb8 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.955580 28654 ts_tablet_manager.cc:1888] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:16.956493 28654 log.cc:1196] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/010d95e45f6e417c95d4d3e6278f4d74
I20240503 02:09:16.956518 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2983e9a39aed45f29aaaca9c5c010bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.956686 28654 ts_tablet_manager.cc:1909] T 010d95e45f6e417c95d4d3e6278f4d74 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.956907 28654 tablet_replica.cc:330] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.956962 28654 raft_consensus.cc:2229] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.956990 28654 raft_consensus.cc:2258] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.957000 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 010d95e45f6e417c95d4d3e6278f4d74 (table default.Table_443f4d53ffca4f209904f7da04bed3be [id=56970436dd49486f86fc2fc41f5de5d7]) successfully deleted
I20240503 02:09:16.957208 28654 ts_tablet_manager.cc:1875] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.957490 28740 raft_consensus.cc:482] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.957523 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 2983e9a39aed45f29aaaca9c5c010bb8 (DEFAULT_TABLE table=default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:16.957538 28740 raft_consensus.cc:504] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.957615 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2983e9a39aed45f29aaaca9c5c010bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:16.957653 28740 leader_election.cc:288] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.957844 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e29e26955d441ef9f7ca908da29ca49" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.957912 27449 raft_consensus.cc:2454] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.957908 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e29e26955d441ef9f7ca908da29ca49" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.957966 27582 raft_consensus.cc:2454] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.958042 27659 leader_election.cc:302] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.958047 28661 ts_tablet_manager.cc:1888] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.958115 28661 log.cc:1196] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a99054ca9753450db1ac75a35ae25e55
I20240503 02:09:16.958130 28740 raft_consensus.cc:2781] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.958182 28740 raft_consensus.cc:482] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.958211 28740 raft_consensus.cc:3037] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.958648 28740 raft_consensus.cc:504] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.958757 28740 leader_election.cc:288] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.958900 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e29e26955d441ef9f7ca908da29ca49" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.958899 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e29e26955d441ef9f7ca908da29ca49" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.958953 27582 raft_consensus.cc:3037] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.958961 27449 raft_consensus.cc:3037] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.959023 28664 tablet_bootstrap.cc:654] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.959071 28654 ts_tablet_manager.cc:1888] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:16.959112 28654 log.cc:1196] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fa960c0281414895a12972c4c1e5269c
I20240503 02:09:16.959281 28654 ts_tablet_manager.cc:1909] T fa960c0281414895a12972c4c1e5269c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.959512 27582 raft_consensus.cc:2454] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.959512 27449 raft_consensus.cc:2454] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.959559 28654 tablet_replica.cc:330] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:16.959618 28654 raft_consensus.cc:2229] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:16.959645 28654 raft_consensus.cc:2258] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:16.959641 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fa960c0281414895a12972c4c1e5269c (table default.Table_bc72842daac14d2cbe4e8d4b52418c8e [id=5737fe41b1164d128a0f6b4b97b69340]) successfully deleted
I20240503 02:09:16.959652 27659 leader_election.cc:302] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.959722 28740 raft_consensus.cc:2781] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.959765 28740 raft_consensus.cc:686] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.959800 28740 consensus_queue.cc:237] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.959884 28661 ts_tablet_manager.cc:1909] T a99054ca9753450db1ac75a35ae25e55 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:16.960179 28654 ts_tablet_manager.cc:1875] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:16.960290 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a99054ca9753450db1ac75a35ae25e55 (table default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]) successfully deleted
I20240503 02:09:16.960629 29566 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.960969 28710 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.960975 28655 tablet_bootstrap.cc:492] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.961262 28655 ts_tablet_manager.cc:1367] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20240503 02:09:16.961450 28655 raft_consensus.cc:348] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.961517 28655 raft_consensus.cc:374] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.961536 28655 raft_consensus.cc:729] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.961571 28655 consensus_queue.cc:260] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.961583 28654 ts_tablet_manager.cc:1888] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:16.961624 28654 log.cc:1196] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a99054ca9753450db1ac75a35ae25e55
I20240503 02:09:16.961663 28655 ts_tablet_manager.cc:1398] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.961740 28655 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.961802 28654 ts_tablet_manager.cc:1909] T a99054ca9753450db1ac75a35ae25e55 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:16.962111 28664 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.962153 28664 ts_tablet_manager.cc:1367] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.962164 28655 tablet_bootstrap.cc:654] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.962234 28664 raft_consensus.cc:348] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.962281 28664 raft_consensus.cc:374] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.962298 28664 raft_consensus.cc:729] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.962334 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a99054ca9753450db1ac75a35ae25e55 (table default.Table_0d3720cfc5c141a4aec2a7fd638195d5 [id=f8a5adcbe89947bfbd664a75b7a4ed63]) successfully deleted
I20240503 02:09:16.962337 28664 consensus_queue.cc:260] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.962428 28664 ts_tablet_manager.cc:1398] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.962488 28664 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:16.962616 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6873697085ec4322a5871dd591dd81dd (table default.Table_5aa184df608d4c2dafb9752f9a6e615e [id=7a6c26f3de2541ebb9d9b8275a66c5a6]). Expected version 1 got 0
I20240503 02:09:16.962705 28664 tablet_bootstrap.cc:654] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.963413 27200 catalog_manager.cc:5484] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.963579 29558 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:16.965832 28656 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.965875 28656 ts_tablet_manager.cc:1367] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20240503 02:09:16.965982 28656 raft_consensus.cc:348] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.966038 28656 raft_consensus.cc:374] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.966056 28656 raft_consensus.cc:729] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.966095 28656 consensus_queue.cc:260] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.966173 28656 ts_tablet_manager.cc:1398] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.966218 28656 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.966619 28656 tablet_bootstrap.cc:654] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.966671 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:16.966854 27848 raft_consensus.cc:482] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.966910 27848 raft_consensus.cc:504] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.967000 27848 leader_election.cc:288] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.967074 28656 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.967111 28656 ts_tablet_manager.cc:1367] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:16.967168 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3297f72c695944debdc8c733280bb86a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.967226 27723 raft_consensus.cc:2454] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.967226 28656 raft_consensus.cc:348] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.967275 28656 raft_consensus.cc:374] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.967291 28656 raft_consensus.cc:729] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.967337 27523 leader_election.cc:302] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.967375 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3297f72c695944debdc8c733280bb86a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.967347 28656 consensus_queue.cc:260] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.967427 27451 raft_consensus.cc:2454] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.967411 27848 raft_consensus.cc:2781] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.967489 28656 ts_tablet_manager.cc:1398] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.967561 28656 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:16.967705 28655 tablet_bootstrap.cc:492] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.967499 27848 raft_consensus.cc:482] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.967710 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:16.967742 27848 raft_consensus.cc:3037] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.967743 28655 ts_tablet_manager.cc:1367] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:16.967868 28656 tablet_bootstrap.cc:654] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.967872 28655 raft_consensus.cc:348] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.967909 28655 raft_consensus.cc:374] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.967921 28655 raft_consensus.cc:729] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.967952 28655 consensus_queue.cc:260] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.968036 28655 ts_tablet_manager.cc:1398] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.968081 28655 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.968402 28655 tablet_bootstrap.cc:654] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.968369 27848 raft_consensus.cc:504] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.968489 27848 leader_election.cc:288] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:16.968700 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3297f72c695944debdc8c733280bb86a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.968756 27723 raft_consensus.cc:3037] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.968760 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3297f72c695944debdc8c733280bb86a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.968812 27451 raft_consensus.cc:3037] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.969405 27723 raft_consensus.cc:2454] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.969461 27451 raft_consensus.cc:2454] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.969571 27523 leader_election.cc:302] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.969604 28664 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:16.969641 28664 ts_tablet_manager.cc:1367] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:16.969645 27848 raft_consensus.cc:2781] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.969691 27848 raft_consensus.cc:686] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.969725 27848 consensus_queue.cc:237] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.969734 28664 raft_consensus.cc:348] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.969794 28664 raft_consensus.cc:374] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.969812 28664 raft_consensus.cc:729] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.969847 28664 consensus_queue.cc:260] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.969923 28664 ts_tablet_manager.cc:1398] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.970278 28656 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:16.970319 28656 ts_tablet_manager.cc:1367] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.970432 28656 raft_consensus.cc:348] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.970482 28656 raft_consensus.cc:374] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.970500 28656 raft_consensus.cc:729] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.970532 28656 consensus_queue.cc:260] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.970611 28656 ts_tablet_manager.cc:1398] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.970646 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:16.971436 27200 catalog_manager.cc:5484] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:16.971590 28655 tablet_bootstrap.cc:492] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.971637 28655 ts_tablet_manager.cc:1367] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:16.971756 28655 raft_consensus.cc:348] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.971814 28655 raft_consensus.cc:374] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.971830 28655 raft_consensus.cc:729] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.971868 28655 consensus_queue.cc:260] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.971980 28655 ts_tablet_manager.cc:1398] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.972034 28655 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.972368 28655 tablet_bootstrap.cc:654] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.973004 28655 tablet_bootstrap.cc:492] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.973053 28655 ts_tablet_manager.cc:1367] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.973186 28655 raft_consensus.cc:348] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.973243 28655 raft_consensus.cc:374] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.973261 28655 raft_consensus.cc:729] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.973296 28655 consensus_queue.cc:260] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.973412 28655 ts_tablet_manager.cc:1398] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:16.973471 28655 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.973793 28655 tablet_bootstrap.cc:654] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.974061 29551 raft_consensus.cc:482] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.974102 29551 raft_consensus.cc:504] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.974128 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:16.974210 29551 leader_election.cc:288] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.974378 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ea8592a1154e4e80f75853e2664e07" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.974438 27723 raft_consensus.cc:2454] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.974511 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ea8592a1154e4e80f75853e2664e07" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.974581 27582 raft_consensus.cc:2454] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:16.974599 27388 leader_election.cc:302] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.974665 29550 raft_consensus.cc:2781] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.974687 29550 raft_consensus.cc:482] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.974704 29550 raft_consensus.cc:3037] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.974776 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:16.975409 29550 raft_consensus.cc:504] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.975510 29550 leader_election.cc:288] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.975637 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ea8592a1154e4e80f75853e2664e07" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.975636 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ea8592a1154e4e80f75853e2664e07" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.975689 27723 raft_consensus.cc:3037] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.975690 27582 raft_consensus.cc:3037] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.976107 27582 raft_consensus.cc:2454] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.976121 27723 raft_consensus.cc:2454] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:16.976258 27386 leader_election.cc:302] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.976300 28655 tablet_bootstrap.cc:492] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.976331 28655 ts_tablet_manager.cc:1367] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:16.976351 29550 raft_consensus.cc:2781] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.976397 29550 raft_consensus.cc:686] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:16.976439 29550 consensus_queue.cc:237] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.976452 28655 raft_consensus.cc:348] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.976754 28655 raft_consensus.cc:374] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.976786 28655 raft_consensus.cc:729] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.976832 28655 consensus_queue.cc:260] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.976954 28655 ts_tablet_manager.cc:1398] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:16.977005 28655 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:16.977382 27202 catalog_manager.cc:5484] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.977500 28655 tablet_bootstrap.cc:654] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:16.978152 28655 tablet_bootstrap.cc:492] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:16.978199 28655 ts_tablet_manager.cc:1367] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:16.978335 28655 raft_consensus.cc:348] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.978394 28655 raft_consensus.cc:374] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:16.978411 28655 raft_consensus.cc:729] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:16.978458 28655 consensus_queue.cc:260] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:16.978545 28655 ts_tablet_manager.cc:1398] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:16.979418 28740 consensus_queue.cc:1035] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.980794 28693 consensus_queue.cc:1035] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.983057 28740 raft_consensus.cc:482] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.983101 28740 raft_consensus.cc:504] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.983186 28740 leader_election.cc:288] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.983351 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d41e4e82931437e808ac39826c285b1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.983354 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d41e4e82931437e808ac39826c285b1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:16.983408 27451 raft_consensus.cc:2454] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.983410 27582 raft_consensus.cc:2454] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:16.983541 27659 leader_election.cc:302] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.983625 28740 raft_consensus.cc:2781] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.983662 28740 raft_consensus.cc:482] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.983678 28740 raft_consensus.cc:3037] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.984113 28740 raft_consensus.cc:504] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.984210 28740 leader_election.cc:288] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:16.984330 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d41e4e82931437e808ac39826c285b1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.984386 27451 raft_consensus.cc:3037] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.984380 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d41e4e82931437e808ac39826c285b1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:16.984457 27582 raft_consensus.cc:3037] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.984941 27451 raft_consensus.cc:2454] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.984966 27582 raft_consensus.cc:2454] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:16.985145 27656 leader_election.cc:302] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.985222 28740 raft_consensus.cc:2781] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.985260 28740 raft_consensus.cc:686] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:16.985292 28740 consensus_queue.cc:237] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.985342 28507 consensus_queue.cc:1035] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.986088 27202 catalog_manager.cc:5484] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:16.986219 28507 raft_consensus.cc:482] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:16.986266 28507 raft_consensus.cc:504] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.986339 28507 leader_election.cc:288] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:16.986433 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab656d127ee405bb11ad94df6eba430" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:16.986505 27449 raft_consensus.cc:2454] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.986693 27524 leader_election.cc:302] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:16.986778 28410 raft_consensus.cc:2781] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:16.986765 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab656d127ee405bb11ad94df6eba430" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:16.986819 28410 raft_consensus.cc:482] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:16.986837 28410 raft_consensus.cc:3037] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.986835 27722 raft_consensus.cc:2454] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:16.987426 28410 raft_consensus.cc:504] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.987506 28410 leader_election.cc:288] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:16.987583 29240 consensus_queue.cc:1035] T 93ad86edb8f646f5b717463b4fae2263 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.987785 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab656d127ee405bb11ad94df6eba430" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:16.987835 27451 raft_consensus.cc:3037] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.988225 28740 consensus_queue.cc:1035] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.988370 27451 raft_consensus.cc:2454] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.988629 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab656d127ee405bb11ad94df6eba430" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:16.988680 27723 raft_consensus.cc:3037] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:16.989209 27723 raft_consensus.cc:2454] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:16.989468 27523 leader_election.cc:302] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:16.989573 29240 raft_consensus.cc:2781] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:16.989609 29240 raft_consensus.cc:686] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:16.989642 29240 consensus_queue.cc:237] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:16.990331 29643 consensus_queue.cc:1035] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:16.991552 27202 catalog_manager.cc:5484] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.003119 29550 consensus_queue.cc:1035] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.003705 28410 raft_consensus.cc:482] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.003752 28410 raft_consensus.cc:504] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.003852 28410 leader_election.cc:288] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.004110 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffac3388531d43d19347097204a8a3d0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.004179 27723 raft_consensus.cc:2454] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.004360 27523 leader_election.cc:302] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.004442 28410 raft_consensus.cc:2781] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.004484 28410 raft_consensus.cc:482] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.004508 28410 raft_consensus.cc:3037] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.004475 28766 consensus_queue.cc:1035] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.004577 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffac3388531d43d19347097204a8a3d0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.004642 27449 raft_consensus.cc:2454] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.005120 28410 raft_consensus.cc:504] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.005244 28410 leader_election.cc:288] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.005403 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffac3388531d43d19347097204a8a3d0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.005462 27449 raft_consensus.cc:3037] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.005748 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffac3388531d43d19347097204a8a3d0" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.005800 27722 raft_consensus.cc:3037] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.006044 27449 raft_consensus.cc:2454] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.006398 27722 raft_consensus.cc:2454] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.006541 27524 leader_election.cc:302] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.006734 28410 raft_consensus.cc:2781] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.006798 28410 raft_consensus.cc:686] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.006839 28410 consensus_queue.cc:237] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.007678 27196 catalog_manager.cc:5484] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.009711 28410 raft_consensus.cc:482] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.009765 28410 raft_consensus.cc:504] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.009860 28410 leader_election.cc:288] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.009968 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1800d45b4f4593aa21127c2fc93f56" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.010025 27449 raft_consensus.cc:2454] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.010087 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1800d45b4f4593aa21127c2fc93f56" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.010134 27722 raft_consensus.cc:2454] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.010149 27524 leader_election.cc:302] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.010347 28410 raft_consensus.cc:2781] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.010380 28410 raft_consensus.cc:482] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.010406 28410 raft_consensus.cc:3037] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.010936 28410 raft_consensus.cc:504] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.011049 28410 leader_election.cc:288] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.011163 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1800d45b4f4593aa21127c2fc93f56" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.011217 27722 raft_consensus.cc:3037] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.011247 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1800d45b4f4593aa21127c2fc93f56" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.011309 27449 raft_consensus.cc:3037] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.011759 27722 raft_consensus.cc:2454] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.011930 27449 raft_consensus.cc:2454] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.012082 27524 leader_election.cc:302] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.012434 28410 raft_consensus.cc:2781] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.012483 28410 raft_consensus.cc:686] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.012518 28410 consensus_queue.cc:237] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.013448 27196 catalog_manager.cc:5484] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.015157 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_229b7e8c38f64c879016be88401b4602" } modify_external_catalogs: true
I20240503 02:09:17.015244 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_229b7e8c38f64c879016be88401b4602" } modify_external_catalogs: true
I20240503 02:09:17.015264 28507 raft_consensus.cc:482] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.015311 28507 raft_consensus.cc:504] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.015388 28507 leader_election.cc:288] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:17.015564 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a549a5309d0451a8b58607ecdc3378a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.015578 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a549a5309d0451a8b58607ecdc3378a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.015635 27449 raft_consensus.cc:2454] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.015635 27722 raft_consensus.cc:2454] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.015743 27524 leader_election.cc:302] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.015817 28507 raft_consensus.cc:2781] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.015858 28507 raft_consensus.cc:482] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.015875 28507 raft_consensus.cc:3037] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.016402 28507 raft_consensus.cc:504] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.016494 28507 leader_election.cc:288] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:17.016640 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a549a5309d0451a8b58607ecdc3378a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.016665 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a549a5309d0451a8b58607ecdc3378a" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.016700 27449 raft_consensus.cc:3037] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.016727 27722 raft_consensus.cc:3037] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.016925 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 69ea8592a1154e4e80f75853e2664e07
I20240503 02:09:17.017159 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ea8592a1154e4e80f75853e2664e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.017163 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ea8592a1154e4e80f75853e2664e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.017232 27449 raft_consensus.cc:2454] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.017242 28654 tablet_replica.cc:330] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.017280 27722 raft_consensus.cc:2454] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.017295 28654 raft_consensus.cc:2229] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.017349 28654 raft_consensus.cc:2258] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.017408 28653 tablet_replica.cc:330] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.017434 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ea8592a1154e4e80f75853e2664e07 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.017452 28653 raft_consensus.cc:2229] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.017449 27524 leader_election.cc:302] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.017491 28653 pending_rounds.cc:62] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:17.017498 28661 tablet_replica.cc:330] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.017510 28653 pending_rounds.cc:68] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420034975895552 op_type: NO_OP noop_request { }
I20240503 02:09:17.017529 28654 ts_tablet_manager.cc:1875] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.017541 28653 raft_consensus.cc:2866] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.017549 28661 raft_consensus.cc:2229] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.017563 28653 raft_consensus.cc:2258] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.017576 28661 raft_consensus.cc:2258] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.017727 28507 raft_consensus.cc:2781] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.017756 28661 ts_tablet_manager.cc:1875] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.017767 28507 raft_consensus.cc:686] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.017812 28507 consensus_queue.cc:237] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.018091 28653 ts_tablet_manager.cc:1875] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.018764 28654 ts_tablet_manager.cc:1888] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.018808 28654 log.cc:1196] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/69ea8592a1154e4e80f75853e2664e07
I20240503 02:09:17.018942 28654 ts_tablet_manager.cc:1909] T 69ea8592a1154e4e80f75853e2664e07 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.019037 28661 ts_tablet_manager.cc:1888] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.019080 28661 log.cc:1196] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/69ea8592a1154e4e80f75853e2664e07
I20240503 02:09:17.019225 28661 ts_tablet_manager.cc:1909] T 69ea8592a1154e4e80f75853e2664e07 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.019387 28653 ts_tablet_manager.cc:1888] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.019425 28653 log.cc:1196] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/69ea8592a1154e4e80f75853e2664e07
I20240503 02:09:17.019588 28653 ts_tablet_manager.cc:1909] T 69ea8592a1154e4e80f75853e2664e07 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.019724 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 69ea8592a1154e4e80f75853e2664e07 (table default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]) successfully deleted
I20240503 02:09:17.020035 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 69ea8592a1154e4e80f75853e2664e07 (table default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]) successfully deleted
I20240503 02:09:17.020195 27196 catalog_manager.cc:5484] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.020859 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 69ea8592a1154e4e80f75853e2664e07 (table default.Table_229b7e8c38f64c879016be88401b4602 [id=bdcfa777b3d3430ab2d9481cd0d358c8]) successfully deleted
I20240503 02:09:17.029192 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_edbab648e55a4187a305627d4d7e00de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.029304 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_edbab648e55a4187a305627d4d7e00de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.033388 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 657eaa8008d34178b52af2ee7af49e03 (DEFAULT_TABLE table=default.Table_edbab648e55a4187a305627d4d7e00de [id=77df8a8a7be54680a39173564721992f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.033442 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 657eaa8008d34178b52af2ee7af49e03 (DEFAULT_TABLE table=default.Table_edbab648e55a4187a305627d4d7e00de [id=77df8a8a7be54680a39173564721992f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.033486 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657eaa8008d34178b52af2ee7af49e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.033515 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657eaa8008d34178b52af2ee7af49e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.033492 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 657eaa8008d34178b52af2ee7af49e03 (DEFAULT_TABLE table=default.Table_edbab648e55a4187a305627d4d7e00de [id=77df8a8a7be54680a39173564721992f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.033576 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657eaa8008d34178b52af2ee7af49e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.034332 28656 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.034576 28664 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.034687 28656 tablet_bootstrap.cc:654] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.034933 28664 tablet_bootstrap.cc:654] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.035166 28656 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.035192 28655 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.035204 28656 ts_tablet_manager.cc:1367] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.035351 28656 raft_consensus.cc:348] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.035399 28656 raft_consensus.cc:374] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.035403 28664 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.035418 28656 raft_consensus.cc:729] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.035456 28664 ts_tablet_manager.cc:1367] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.035460 28656 consensus_queue.cc:260] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.035473 28655 tablet_bootstrap.cc:654] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.035550 28656 ts_tablet_manager.cc:1398] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.035599 28664 raft_consensus.cc:348] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.035650 28664 raft_consensus.cc:374] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.035666 28664 raft_consensus.cc:729] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.035701 28664 consensus_queue.cc:260] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.035797 28664 ts_tablet_manager.cc:1398] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.036221 28655 tablet_bootstrap.cc:492] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.036257 28655 ts_tablet_manager.cc:1367] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.036362 28655 raft_consensus.cc:348] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.036396 28655 raft_consensus.cc:374] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.036406 28655 raft_consensus.cc:729] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.036433 28655 consensus_queue.cc:260] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.036509 28655 ts_tablet_manager.cc:1398] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.043944 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d895ab85cb064b40bdf6cebed80debbe" } modify_external_catalogs: true
I20240503 02:09:17.044003 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d895ab85cb064b40bdf6cebed80debbe" } modify_external_catalogs: true
I20240503 02:09:17.044754 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet dde17e7bed0e487189e33d614ab11a9e
I20240503 02:09:17.044986 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet dde17e7bed0e487189e33d614ab11a9e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.044999 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet dde17e7bed0e487189e33d614ab11a9e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.044996 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet dde17e7bed0e487189e33d614ab11a9e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.045073 28661 tablet_replica.cc:330] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.045125 28661 raft_consensus.cc:2229] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.045163 28661 raft_consensus.cc:2258] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.045264 28661 ts_tablet_manager.cc:1875] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.046181 28653 tablet_replica.cc:330] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.046243 28653 raft_consensus.cc:2229] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.046305 28653 raft_consensus.cc:2258] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.046389 28661 ts_tablet_manager.cc:1888] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.046442 28661 log.cc:1196] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/dde17e7bed0e487189e33d614ab11a9e
I20240503 02:09:17.046511 28653 ts_tablet_manager.cc:1875] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.046612 28661 ts_tablet_manager.cc:1909] T dde17e7bed0e487189e33d614ab11a9e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.046849 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:17.047616 28653 ts_tablet_manager.cc:1888] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.047672 28653 log.cc:1196] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/dde17e7bed0e487189e33d614ab11a9e
I20240503 02:09:17.047875 28653 ts_tablet_manager.cc:1909] T dde17e7bed0e487189e33d614ab11a9e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.048082 28654 tablet_replica.cc:330] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.048090 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e3f36c99ead44db1a3689a0c14c0dd02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:17.048151 28654 raft_consensus.cc:2229] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
W20240503 02:09:17.048183 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3f36c99ead44db1a3689a0c14c0dd02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.048210 28654 raft_consensus.cc:2258] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.048321 28654 ts_tablet_manager.cc:1875] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.049410 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:17.049623 28654 ts_tablet_manager.cc:1888] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.049654 28654 log.cc:1196] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/dde17e7bed0e487189e33d614ab11a9e
I20240503 02:09:17.049834 28654 ts_tablet_manager.cc:1909] T dde17e7bed0e487189e33d614ab11a9e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.050226 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet dde17e7bed0e487189e33d614ab11a9e (table default.Table_d895ab85cb064b40bdf6cebed80debbe [id=41e048c6e5644782afa7023a8a808b02]) successfully deleted
I20240503 02:09:17.050632 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1a5c30890d491f979cdb870dbdcf76 (DEFAULT_TABLE table=default.Table_e3f36c99ead44db1a3689a0c14c0dd02 [id=7446db7cc6e246e2a66677ebabfa1054]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.050684 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1a5c30890d491f979cdb870dbdcf76 (DEFAULT_TABLE table=default.Table_e3f36c99ead44db1a3689a0c14c0dd02 [id=7446db7cc6e246e2a66677ebabfa1054]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.050730 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1a5c30890d491f979cdb870dbdcf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.050750 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1a5c30890d491f979cdb870dbdcf76 (DEFAULT_TABLE table=default.Table_e3f36c99ead44db1a3689a0c14c0dd02 [id=7446db7cc6e246e2a66677ebabfa1054]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.050786 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1a5c30890d491f979cdb870dbdcf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.050830 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1a5c30890d491f979cdb870dbdcf76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.051512 28664 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.051764 28655 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.051893 28664 tablet_bootstrap.cc:654] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.052109 28655 tablet_bootstrap.cc:654] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.052275 28656 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.052361 28664 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.052395 28664 ts_tablet_manager.cc:1367] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.052512 28655 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.052541 28655 ts_tablet_manager.cc:1367] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.052524 28664 raft_consensus.cc:348] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.052582 28664 raft_consensus.cc:374] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.052600 28664 raft_consensus.cc:729] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.052623 28656 tablet_bootstrap.cc:654] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.052634 28655 raft_consensus.cc:348] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.052667 28655 raft_consensus.cc:374] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.052647 28664 consensus_queue.cc:260] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.052678 28655 raft_consensus.cc:729] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.052719 28655 consensus_queue.cc:260] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.052770 28664 ts_tablet_manager.cc:1398] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.052803 28655 ts_tablet_manager.cc:1398] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.053427 28656 tablet_bootstrap.cc:492] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.053471 28656 ts_tablet_manager.cc:1367] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.053603 28656 raft_consensus.cc:348] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.053658 28656 raft_consensus.cc:374] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.053678 28656 raft_consensus.cc:729] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.053711 28656 consensus_queue.cc:260] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.053789 28656 ts_tablet_manager.cc:1398] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.054277 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.054369 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.055090 28767 consensus_queue.cc:1035] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.055155 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bc57d1eab70844229eae43ec947567af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.055259 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc57d1eab70844229eae43ec947567af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.057080 28410 consensus_queue.cc:1035] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.057487 28409 consensus_queue.cc:1035] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.057940 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8f3809378f444386cdd868e6db4c04 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.057960 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8f3809378f444386cdd868e6db4c04 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.058040 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8f3809378f444386cdd868e6db4c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.058046 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8f3809378f444386cdd868e6db4c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.058974 28656 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.059310 28656 tablet_bootstrap.cc:654] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.059599 28664 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.059762 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7a8f3809378f444386cdd868e6db4c04 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.059844 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a8f3809378f444386cdd868e6db4c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.060432 28656 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.060468 28656 ts_tablet_manager.cc:1367] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:17.060532 28767 raft_consensus.cc:482] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.060570 28767 raft_consensus.cc:504] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.060593 28656 raft_consensus.cc:348] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.060654 28656 raft_consensus.cc:374] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.060673 28656 raft_consensus.cc:729] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.060668 28767 leader_election.cc:288] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.060707 28656 consensus_queue.cc:260] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.060794 28656 ts_tablet_manager.cc:1398] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.060943 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cd092cf71e47fcad1f2f9091953141" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.061009 27588 raft_consensus.cc:2454] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.061082 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cd092cf71e47fcad1f2f9091953141" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.061131 27723 raft_consensus.cc:2454] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.061132 27386 leader_election.cc:302] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.061178 28655 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.061203 28767 raft_consensus.cc:2781] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.061226 28767 raft_consensus.cc:482] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.061242 28767 raft_consensus.cc:3037] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.061785 28655 tablet_bootstrap.cc:654] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.061831 28767 raft_consensus.cc:504] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.061955 28767 leader_election.cc:288] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.062072 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cd092cf71e47fcad1f2f9091953141" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.062125 27723 raft_consensus.cc:3037] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.062109 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53cd092cf71e47fcad1f2f9091953141" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.062155 27582 raft_consensus.cc:3037] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.062520 28664 tablet_bootstrap.cc:654] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.062716 27582 raft_consensus.cc:2454] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.062716 27723 raft_consensus.cc:2454] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.063089 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 43948f8039d948efbf1dcde10e012be4 (DEFAULT_TABLE table=default.Table_bc57d1eab70844229eae43ec947567af [id=9b39c4c4d8d14466bef80236d3908b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.063175 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43948f8039d948efbf1dcde10e012be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.063179 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 43948f8039d948efbf1dcde10e012be4 (DEFAULT_TABLE table=default.Table_bc57d1eab70844229eae43ec947567af [id=9b39c4c4d8d14466bef80236d3908b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.063261 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43948f8039d948efbf1dcde10e012be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.063431 27848 consensus_queue.cc:1035] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.063493 27388 leader_election.cc:302] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.063609 28767 raft_consensus.cc:2781] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.063644 28767 raft_consensus.cc:686] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.063678 28767 consensus_queue.cc:237] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.063892 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 43948f8039d948efbf1dcde10e012be4 (DEFAULT_TABLE table=default.Table_bc57d1eab70844229eae43ec947567af [id=9b39c4c4d8d14466bef80236d3908b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.063966 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43948f8039d948efbf1dcde10e012be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.064162 28656 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.064479 28656 tablet_bootstrap.cc:654] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.064632 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_956e18a0f62f41ad8a03bc02f603261b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.064728 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_956e18a0f62f41ad8a03bc02f603261b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.066308 28664 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.066350 28664 ts_tablet_manager.cc:1367] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:17.066391 28655 tablet_bootstrap.cc:492] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.066431 28655 ts_tablet_manager.cc:1367] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:17.066548 28664 raft_consensus.cc:348] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.066589 28664 raft_consensus.cc:374] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.066560 28655 raft_consensus.cc:348] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.066601 28664 raft_consensus.cc:729] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.066610 28655 raft_consensus.cc:374] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.066630 28655 raft_consensus.cc:729] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.066628 28664 consensus_queue.cc:260] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.066669 28655 consensus_queue.cc:260] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.066709 28664 ts_tablet_manager.cc:1398] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.066749 28655 ts_tablet_manager.cc:1398] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.066783 28664 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.066803 28655 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.067123 28664 tablet_bootstrap.cc:654] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.067137 28655 tablet_bootstrap.cc:654] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.068015 28664 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.068058 28664 ts_tablet_manager.cc:1367] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.068078 27202 catalog_manager.cc:5484] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.068166 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:17.068171 28664 raft_consensus.cc:348] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.068220 28664 raft_consensus.cc:374] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.068236 28664 raft_consensus.cc:729] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.068269 28664 consensus_queue.cc:260] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.068368 28664 ts_tablet_manager.cc:1398] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.068539 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 58311afeda3d408f9a387c6ec2398e3c (DEFAULT_TABLE table=default.Table_956e18a0f62f41ad8a03bc02f603261b [id=401e087f4afc46eda0d0f47b661c7f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.068557 28655 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.068594 28655 ts_tablet_manager.cc:1367] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:17.068619 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58311afeda3d408f9a387c6ec2398e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.068711 28655 raft_consensus.cc:348] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.068761 28655 raft_consensus.cc:374] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.068778 28655 raft_consensus.cc:729] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.068820 28655 consensus_queue.cc:260] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.068912 28655 ts_tablet_manager.cc:1398] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.068984 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 58311afeda3d408f9a387c6ec2398e3c (DEFAULT_TABLE table=default.Table_956e18a0f62f41ad8a03bc02f603261b [id=401e087f4afc46eda0d0f47b661c7f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.069056 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58311afeda3d408f9a387c6ec2398e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.069700 28664 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.070034 28664 tablet_bootstrap.cc:654] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.070106 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 58311afeda3d408f9a387c6ec2398e3c (DEFAULT_TABLE table=default.Table_956e18a0f62f41ad8a03bc02f603261b [id=401e087f4afc46eda0d0f47b661c7f51]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.070186 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58311afeda3d408f9a387c6ec2398e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.070536 28655 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.070856 28664 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.070858 28655 tablet_bootstrap.cc:654] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.070892 28664 ts_tablet_manager.cc:1367] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.071004 28664 raft_consensus.cc:348] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071039 28664 raft_consensus.cc:374] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.071049 28664 raft_consensus.cc:729] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.071080 28664 consensus_queue.cc:260] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071203 28664 ts_tablet_manager.cc:1398] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.071501 28655 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.071539 28655 ts_tablet_manager.cc:1367] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.071651 28656 tablet_bootstrap.cc:492] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.071646 28655 raft_consensus.cc:348] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071681 28656 ts_tablet_manager.cc:1367] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:17.071693 28655 raft_consensus.cc:374] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.071712 28655 raft_consensus.cc:729] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.071764 28655 consensus_queue.cc:260] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071768 28656 raft_consensus.cc:348] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071810 28656 raft_consensus.cc:374] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.071830 28656 raft_consensus.cc:729] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.071803 29224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:17.071854 28655 ts_tablet_manager.cc:1398] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.071866 29224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:17.071862 28656 consensus_queue.cc:260] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.071935 28656 ts_tablet_manager.cc:1398] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.071995 28656 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.072299 28656 tablet_bootstrap.cc:654] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.072813 28656 tablet_bootstrap.cc:492] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.072870 28656 ts_tablet_manager.cc:1367] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.072980 28656 raft_consensus.cc:348] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.073050 28656 raft_consensus.cc:374] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.073067 28656 raft_consensus.cc:729] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.073098 28656 consensus_queue.cc:260] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.073179 28656 ts_tablet_manager.cc:1398] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:17.073422 27188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.117.62:37529: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20240503 02:09:17.075007 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1513d47f8f1848448db5a23d72c63c8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.075103 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1513d47f8f1848448db5a23d72c63c8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.075177 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_057771b8908c4df3be472ab48352e93a" } new_table_name: "default.Table_ee0946e70ae74889ace95c7aec71f6d2" modify_external_catalogs: true
I20240503 02:09:17.075187 27202 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f272f6f0eb3846598cece6afcc83681f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.075409 27202 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f272f6f0eb3846598cece6afcc83681f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.077198 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcd0014c93f4a3296e33c7d3221e06e (DEFAULT_TABLE table=default.Table_1513d47f8f1848448db5a23d72c63c8a [id=e2143b8ffeae471d93521e4896bb63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.077283 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcd0014c93f4a3296e33c7d3221e06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.077550 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcd0014c93f4a3296e33c7d3221e06e (DEFAULT_TABLE table=default.Table_1513d47f8f1848448db5a23d72c63c8a [id=e2143b8ffeae471d93521e4896bb63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.077636 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcd0014c93f4a3296e33c7d3221e06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.077694 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcd0014c93f4a3296e33c7d3221e06e (DEFAULT_TABLE table=default.Table_1513d47f8f1848448db5a23d72c63c8a [id=e2143b8ffeae471d93521e4896bb63a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.077775 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcd0014c93f4a3296e33c7d3221e06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.077862 29566 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.078378 29558 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.078603 28664 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.078918 28664 tablet_bootstrap.cc:654] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.079597 28710 tablet.cc:1722] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.079643 28656 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.079959 28656 tablet_bootstrap.cc:654] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.080190 28655 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.080435 28656 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.080471 28656 ts_tablet_manager.cc:1367] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.080603 28655 tablet_bootstrap.cc:654] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.080579 28656 raft_consensus.cc:348] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.080627 28656 raft_consensus.cc:374] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.080646 28656 raft_consensus.cc:729] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.080685 28656 consensus_queue.cc:260] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.080808 28656 ts_tablet_manager.cc:1398] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.081483 28655 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.081493 28664 tablet_bootstrap.cc:492] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.081526 28655 ts_tablet_manager.cc:1367] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.081535 28664 ts_tablet_manager.cc:1367] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.081646 28655 raft_consensus.cc:348] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.081645 28664 raft_consensus.cc:348] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.081692 28655 raft_consensus.cc:374] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.081692 28664 raft_consensus.cc:374] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.081710 28655 raft_consensus.cc:729] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.081710 28664 raft_consensus.cc:729] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.081746 28655 consensus_queue.cc:260] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.081774 28664 consensus_queue.cc:260] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.081789 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0ecd7a67fd4818b25696395a854baf (DEFAULT_TABLE table=default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.081841 28655 ts_tablet_manager.cc:1398] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.081859 28664 ts_tablet_manager.cc:1398] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.081874 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0ecd7a67fd4818b25696395a854baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.082274 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0ecd7a67fd4818b25696395a854baf (DEFAULT_TABLE table=default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.082365 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0ecd7a67fd4818b25696395a854baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.082911 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 6a0ecd7a67fd4818b25696395a854baf (DEFAULT_TABLE table=default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.082994 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a0ecd7a67fd4818b25696395a854baf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.083703 28656 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.083669 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2b3a7371a1ac4f049968c590a81af503"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.083760 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b3a7371a1ac4f049968c590a81af503 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.083982 28655 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.084048 28656 tablet_bootstrap.cc:654] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.084064 28409 raft_consensus.cc:482] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.084100 28409 raft_consensus.cc:504] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.084199 28409 leader_election.cc:288] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.084249 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1] alter complete (version 2)
I20240503 02:09:17.084342 28655 tablet_bootstrap.cc:654] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.084409 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837debf700f74dc680eb88168aeac413" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.084482 27722 raft_consensus.cc:2454] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.084612 27388 leader_election.cc:302] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.084623 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837debf700f74dc680eb88168aeac413" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.084679 27582 raft_consensus.cc:2454] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.084679 28664 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.084698 28409 raft_consensus.cc:2781] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.084723 28409 raft_consensus.cc:482] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.084739 28409 raft_consensus.cc:3037] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.085080 28664 tablet_bootstrap.cc:654] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.085551 28664 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.085594 28664 ts_tablet_manager.cc:1367] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.085727 28664 raft_consensus.cc:348] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.085781 28664 raft_consensus.cc:374] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.085798 28664 raft_consensus.cc:729] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.085765 28409 raft_consensus.cc:504] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.085830 28664 consensus_queue.cc:260] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.085863 28409 leader_election.cc:288] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.085906 28664 ts_tablet_manager.cc:1398] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.086217 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837debf700f74dc680eb88168aeac413" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.086279 27582 raft_consensus.cc:3037] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.086723 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837debf700f74dc680eb88168aeac413" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.086781 27722 raft_consensus.cc:3037] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.086864 27582 raft_consensus.cc:2454] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.087117 27386 leader_election.cc:302] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.087183 28409 raft_consensus.cc:2781] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.087213 28409 raft_consensus.cc:686] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.087260 28409 consensus_queue.cc:237] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.087365 27722 raft_consensus.cc:2454] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.087843 28655 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.087885 28655 ts_tablet_manager.cc:1367] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.088014 28655 raft_consensus.cc:348] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.088078 28655 raft_consensus.cc:374] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.088097 28655 raft_consensus.cc:729] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.088124 28655 consensus_queue.cc:260] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.088199 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 4c14768b631a423084b6f43203a8832e (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.088291 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c14768b631a423084b6f43203a8832e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.088481 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 4c14768b631a423084b6f43203a8832e (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.088567 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c14768b631a423084b6f43203a8832e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.088573 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_92bbec1584154fe7b4840c082ba218ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.088651 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92bbec1584154fe7b4840c082ba218ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.088937 27196 catalog_manager.cc:5484] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.089243 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 4c14768b631a423084b6f43203a8832e (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.089355 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c14768b631a423084b6f43203a8832e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.088204 28655 ts_tablet_manager.cc:1398] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.089896 28655 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.089927 28656 tablet_bootstrap.cc:492] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.089972 28656 ts_tablet_manager.cc:1367] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.090062 28656 raft_consensus.cc:348] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.090094 28656 raft_consensus.cc:374] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.090106 28656 raft_consensus.cc:729] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.090135 28656 consensus_queue.cc:260] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.090210 28656 ts_tablet_manager.cc:1398] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.090219 28655 tablet_bootstrap.cc:654] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.090255 28656 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.090464 27092 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskpsd1k2/build/release/bin/kudu
/tmp/dist-test-taskpsd1k2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--webserver_interface=127.26.117.62
--webserver_port=40493
--builtin_ntp_servers=127.26.117.20:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20240503 02:09:17.090559 28656 tablet_bootstrap.cc:654] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.091672 28656 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.091714 28656 ts_tablet_manager.cc:1367] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.091033 28664 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.091833 28656 raft_consensus.cc:348] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.091882 28656 raft_consensus.cc:374] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.091899 28656 raft_consensus.cc:729] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.091933 28656 consensus_queue.cc:260] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.092020 28656 ts_tablet_manager.cc:1398] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.092204 28664 tablet_bootstrap.cc:654] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.093194 28767 consensus_queue.cc:1035] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.093838 27848 consensus_queue.cc:1035] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.094658 28664 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.094707 28664 ts_tablet_manager.cc:1367] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.094835 28664 raft_consensus.cc:348] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.094903 28664 raft_consensus.cc:374] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.094928 28664 raft_consensus.cc:729] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.094962 28664 consensus_queue.cc:260] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.095052 28664 ts_tablet_manager.cc:1398] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.095988 28774 consensus_queue.cc:1035] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.095988 27202 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8490179685754442b955ed419b307ac6" } modify_external_catalogs: true
I20240503 02:09:17.096058 27202 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8490179685754442b955ed419b307ac6" } modify_external_catalogs: true
I20240503 02:09:17.096475 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 0182ed8465eb40238a7efc2850f18173 (DEFAULT_TABLE table=default.Table_92bbec1584154fe7b4840c082ba218ce [id=4735b72d84f0499bb757629a3788ff9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.096563 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0182ed8465eb40238a7efc2850f18173. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.096590 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 0182ed8465eb40238a7efc2850f18173 (DEFAULT_TABLE table=default.Table_92bbec1584154fe7b4840c082ba218ce [id=4735b72d84f0499bb757629a3788ff9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.096673 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0182ed8465eb40238a7efc2850f18173. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.097258 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 0182ed8465eb40238a7efc2850f18173 (DEFAULT_TABLE table=default.Table_92bbec1584154fe7b4840c082ba218ce [id=4735b72d84f0499bb757629a3788ff9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.097355 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0182ed8465eb40238a7efc2850f18173. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.097712 28664 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.098104 28664 tablet_bootstrap.cc:654] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.098295 28656 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.098454 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_48c8a8213fc54f158347e24c2f4bfe0f" } new_table_name: "default.Table_0bcd732b52b74ae6a8101e4925faada8" modify_external_catalogs: true
I20240503 02:09:17.098632 28664 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.098667 28664 ts_tablet_manager.cc:1367] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.098776 28656 tablet_bootstrap.cc:654] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.098801 28664 raft_consensus.cc:348] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.098851 28664 raft_consensus.cc:374] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.098870 28664 raft_consensus.cc:729] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.098917 28664 consensus_queue.cc:260] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.098995 28664 ts_tablet_manager.cc:1398] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.099618 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ffac3388531d43d19347097204a8a3d0
I20240503 02:09:17.099769 29643 raft_consensus.cc:482] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.099828 29643 raft_consensus.cc:504] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.099915 29643 leader_election.cc:288] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.100144 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657eaa8008d34178b52af2ee7af49e03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.100203 27449 raft_consensus.cc:2454] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.100284 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet ffac3388531d43d19347097204a8a3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.100414 28654 tablet_replica.cc:330] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.100483 28654 raft_consensus.cc:2229] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.100519 28654 pending_rounds.cc:62] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:17.100535 28654 pending_rounds.cc:68] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420035100393472 op_type: NO_OP noop_request { }
I20240503 02:09:17.100564 28654 raft_consensus.cc:2866] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.100584 28654 raft_consensus.cc:2258] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.100677 29241 consensus_queue.cc:1035] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.101148 27659 leader_election.cc:302] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.101225 27853 raft_consensus.cc:2781] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.101250 27853 raft_consensus.cc:482] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.101267 27853 raft_consensus.cc:3037] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.101552 28655 tablet_bootstrap.cc:492] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.101583 28655 ts_tablet_manager.cc:1367] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:17.101670 28655 raft_consensus.cc:348] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.101701 28655 raft_consensus.cc:374] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.101711 28655 raft_consensus.cc:729] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.101742 28655 consensus_queue.cc:260] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.101824 28655 ts_tablet_manager.cc:1398] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.101902 27853 raft_consensus.cc:504] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.101938 28655 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.101995 27853 leader_election.cc:288] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.102329 28655 tablet_bootstrap.cc:654] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.102435 28654 ts_tablet_manager.cc:1875] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.103690 28654 ts_tablet_manager.cc:1888] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.103732 28654 log.cc:1196] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ffac3388531d43d19347097204a8a3d0
I20240503 02:09:17.103844 28410 raft_consensus.cc:482] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.103838 28767 consensus_queue.cc:1035] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.103879 28410 raft_consensus.cc:504] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.103911 28654 ts_tablet_manager.cc:1909] T ffac3388531d43d19347097204a8a3d0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.103966 28410 leader_election.cc:288] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.104023 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet ffac3388531d43d19347097204a8a3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.104087 28653 tablet_replica.cc:330] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.104131 28653 raft_consensus.cc:2229] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.104161 28653 raft_consensus.cc:2258] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.104154 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0ecd7a67fd4818b25696395a854baf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.104174 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0ecd7a67fd4818b25696395a854baf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.104212 27723 raft_consensus.cc:2454] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.104223 27449 raft_consensus.cc:2454] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.104321 27524 leader_election.cc:302] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.104352 28653 ts_tablet_manager.cc:1875] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.104460 28507 raft_consensus.cc:2781] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.104493 28507 raft_consensus.cc:482] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.104509 28507 raft_consensus.cc:3037] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.105093 28507 raft_consensus.cc:504] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.105193 28507 leader_election.cc:288] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.105473 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0ecd7a67fd4818b25696395a854baf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.105497 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet ffac3388531d43d19347097204a8a3d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.105530 27449 raft_consensus.cc:3037] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.105610 28661 tablet_replica.cc:330] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.105656 28661 raft_consensus.cc:2229] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.105688 28661 raft_consensus.cc:2258] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.105999 28661 ts_tablet_manager.cc:1875] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.106060 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657eaa8008d34178b52af2ee7af49e03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.106117 27451 raft_consensus.cc:3037] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.106168 27449 raft_consensus.cc:2454] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.106395 29558 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.106658 27451 raft_consensus.cc:2454] T 657eaa8008d34178b52af2ee7af49e03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.107347 27659 leader_election.cc:302] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.107417 28740 consensus_queue.cc:1035] T 5b97787b7d314c39b4ea9e599dbe692e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.107482 28740 raft_consensus.cc:2781] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.107540 28740 raft_consensus.cc:686] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.107570 28740 consensus_queue.cc:237] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.107620 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a2a06fe94ffe4114ac30f51536351ada"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.107699 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2a06fe94ffe4114ac30f51536351ada in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.107712 28653 ts_tablet_manager.cc:1888] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.107753 28653 log.cc:1196] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ffac3388531d43d19347097204a8a3d0
I20240503 02:09:17.107893 28653 ts_tablet_manager.cc:1909] T ffac3388531d43d19347097204a8a3d0 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.107988 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a0ecd7a67fd4818b25696395a854baf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.108048 27723 raft_consensus.cc:3037] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.108429 28710 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.108616 27723 raft_consensus.cc:2454] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.108942 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_49384112661b4fbba2fbfedd6927a068" } new_table_name: "default.Table_405719b600a24ad69a35850163c91c9c" modify_external_catalogs: true
I20240503 02:09:17.109019 27523 leader_election.cc:302] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.109088 28410 raft_consensus.cc:2781] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.109119 28410 raft_consensus.cc:686] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.109150 28410 consensus_queue.cc:237] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.109308 28410 raft_consensus.cc:482] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.109361 28410 raft_consensus.cc:504] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.109421 28766 raft_consensus.cc:482] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.109438 28410 leader_election.cc:288] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.109467 28766 raft_consensus.cc:504] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.109575 28766 leader_election.cc:288] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.109688 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcd0014c93f4a3296e33c7d3221e06e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.109751 27723 raft_consensus.cc:2454] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.109783 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ffac3388531d43d19347097204a8a3d0 (table default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]) successfully deleted
I20240503 02:09:17.109840 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_666cd538e690425ba8746112d159a266" } new_table_name: "default.Table_a919652e893b4243a0633f5b0fd85a04" modify_external_catalogs: true
I20240503 02:09:17.110024 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1a5c30890d491f979cdb870dbdcf76" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.110076 27588 raft_consensus.cc:2454] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.110149 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657eaa8008d34178b52af2ee7af49e03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.110204 27587 raft_consensus.cc:3037] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.110492 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657eaa8008d34178b52af2ee7af49e03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.110751 27523 leader_election.cc:302] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.110798 27587 raft_consensus.cc:2454] T 657eaa8008d34178b52af2ee7af49e03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.110827 28410 raft_consensus.cc:2781] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.110850 28410 raft_consensus.cc:482] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.110867 28410 raft_consensus.cc:3037] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.110877 27386 leader_election.cc:302] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.111131 27196 catalog_manager.cc:5484] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.111307 28766 raft_consensus.cc:2781] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.111341 28766 raft_consensus.cc:482] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.111358 28766 raft_consensus.cc:3037] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.111438 28410 raft_consensus.cc:504] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.111546 28410 leader_election.cc:288] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.111727 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1a5c30890d491f979cdb870dbdcf76" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.111778 27723 raft_consensus.cc:2454] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.111956 28661 ts_tablet_manager.cc:1888] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.111994 28661 log.cc:1196] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ffac3388531d43d19347097204a8a3d0
I20240503 02:09:17.111969 28766 raft_consensus.cc:504] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.112076 28766 leader_election.cc:288] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.112146 28661 ts_tablet_manager.cc:1909] T ffac3388531d43d19347097204a8a3d0 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.112303 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcd0014c93f4a3296e33c7d3221e06e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.112358 27718 raft_consensus.cc:3037] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.112772 27718 raft_consensus.cc:2454] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.112927 27523 leader_election.cc:302] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.112995 28507 raft_consensus.cc:2781] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.113037 28507 raft_consensus.cc:686] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.113067 28507 consensus_queue.cc:237] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.113126 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ffac3388531d43d19347097204a8a3d0 (table default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]) successfully deleted
I20240503 02:09:17.113385 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1a5c30890d491f979cdb870dbdcf76" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.113437 27718 raft_consensus.cc:3037] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.113833 27718 raft_consensus.cc:2454] T 4a1a5c30890d491f979cdb870dbdcf76 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.113981 27388 leader_election.cc:302] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.114068 28766 raft_consensus.cc:2781] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.114097 28766 raft_consensus.cc:686] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.114127 28766 consensus_queue.cc:237] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.114848 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1a5c30890d491f979cdb870dbdcf76" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.114907 27583 raft_consensus.cc:3037] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.114930 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcd0014c93f4a3296e33c7d3221e06e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.115018 27450 raft_consensus.cc:2454] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.115391 29566 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.115374 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcd0014c93f4a3296e33c7d3221e06e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.115443 27450 raft_consensus.cc:3037] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.115475 27583 raft_consensus.cc:2454] T 4a1a5c30890d491f979cdb870dbdcf76 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.116006 27450 raft_consensus.cc:2454] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
W20240503 02:09:17.116294 27202 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fcc99e566bc24ad6a98d20dc61bcfa6d consensus state has no leader
I20240503 02:09:17.116555 27198 catalog_manager.cc:5484] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.117615 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ffac3388531d43d19347097204a8a3d0 (table default.Table_8490179685754442b955ed419b307ac6 [id=0b7bf254d6ab4408aab3835c2a5c6f76]) successfully deleted
I20240503 02:09:17.117774 27196 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_0bcd732b52b74ae6a8101e4925faada8 [id=c10aff1a98f64f13a7ffb57bf8a6e815] alter complete (version 2)
I20240503 02:09:17.117834 28656 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.117866 28656 ts_tablet_manager.cc:1367] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.000s
I20240503 02:09:17.117976 28656 raft_consensus.cc:348] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.118028 28656 raft_consensus.cc:374] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.118047 28656 raft_consensus.cc:729] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.118081 28656 consensus_queue.cc:260] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.118160 28656 ts_tablet_manager.cc:1398] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.119305 28710 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.119799 28410 raft_consensus.cc:482] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.119845 28655 tablet_bootstrap.cc:492] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.119846 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet d5377c9b1e6c419e8f8a3ed4f0855c06 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.119844 28410 raft_consensus.cc:504] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.119889 28655 ts_tablet_manager.cc:1367] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.000s
I20240503 02:09:17.119937 28410 leader_election.cc:288] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.120002 28655 raft_consensus.cc:348] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.120052 28655 raft_consensus.cc:374] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.120070 28655 raft_consensus.cc:729] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.120103 28655 consensus_queue.cc:260] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.120177 28655 ts_tablet_manager.cc:1398] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.120258 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcc99e566bc24ad6a98d20dc61bcfa6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.120311 27450 raft_consensus.cc:2454] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.120424 27524 leader_election.cc:302] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.120596 28410 raft_consensus.cc:2781] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.120630 28410 raft_consensus.cc:482] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.120646 28410 raft_consensus.cc:3037] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.120746 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcc99e566bc24ad6a98d20dc61bcfa6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.120802 27718 raft_consensus.cc:2454] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.121048 29558 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.121223 28410 raft_consensus.cc:504] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.121356 28410 leader_election.cc:288] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.121757 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcc99e566bc24ad6a98d20dc61bcfa6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.121815 27718 raft_consensus.cc:3037] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.122059 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcc99e566bc24ad6a98d20dc61bcfa6d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.122113 27450 raft_consensus.cc:3037] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.122300 27718 raft_consensus.cc:2454] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.122668 27450 raft_consensus.cc:2454] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.123366 27524 leader_election.cc:302] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.123653 28410 raft_consensus.cc:2781] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.123697 28410 raft_consensus.cc:686] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.123726 28410 consensus_queue.cc:237] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.123777 29566 tablet.cc:1722] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.124076 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet f2a07194e9874f7f9df96fc754e25779 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.124321 27195 catalog_manager.cc:5484] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.124596 27562 tablet_service.cc:1467] Processing CreateTablet for tablet cb958efd36594d5390ae617940aed908 (DEFAULT_TABLE table=default.Table_a2a06fe94ffe4114ac30f51536351ada [id=4a7f33c92cef4038a1fe313b4c461812]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.124678 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb958efd36594d5390ae617940aed908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.124703 27196 catalog_manager.cc:5484] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.125581 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d5377c9b1e6c419e8f8a3ed4f0855c06
I20240503 02:09:17.125671 27198 catalog_manager.cc:6554] ReplaceTablet: tablet d5377c9b1e6c419e8f8a3ed4f0855c06 deleted and replaced by tablet 051d0dfb8e744d0ca7044f0c03835a90
I20240503 02:09:17.125831 28656 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.126108 27201 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e] alter complete (version 2)
I20240503 02:09:17.126188 28656 tablet_bootstrap.cc:654] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.126374 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 with delete_type TABLET_DATA_DELETED (replaced by tablet 051d0dfb8e744d0ca7044f0c03835a90) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.126446 28653 tablet_replica.cc:330] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.126484 27427 tablet_service.cc:1467] Processing CreateTablet for tablet cb958efd36594d5390ae617940aed908 (DEFAULT_TABLE table=default.Table_a2a06fe94ffe4114ac30f51536351ada [id=4a7f33c92cef4038a1fe313b4c461812]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.126518 28653 raft_consensus.cc:2229] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.126554 28653 raft_consensus.cc:2258] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.126569 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb958efd36594d5390ae617940aed908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.126631 27703 tablet_service.cc:1467] Processing CreateTablet for tablet cb958efd36594d5390ae617940aed908 (DEFAULT_TABLE table=default.Table_a2a06fe94ffe4114ac30f51536351ada [id=4a7f33c92cef4038a1fe313b4c461812]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.126708 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 with delete_type TABLET_DATA_DELETED (replaced by tablet 051d0dfb8e744d0ca7044f0c03835a90) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.126727 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb958efd36594d5390ae617940aed908. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.126720 28653 ts_tablet_manager.cc:1875] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.127605 28654 tablet_replica.cc:330] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.127614 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b8c8954eca3b491181c678ac0c417dff" } new_table_name: "default.Table_b1dcd1438e66456ea9ed0228abb696da" modify_external_catalogs: true
I20240503 02:09:17.127638 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet d5377c9b1e6c419e8f8a3ed4f0855c06 with delete_type TABLET_DATA_DELETED (replaced by tablet 051d0dfb8e744d0ca7044f0c03835a90) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.127660 28654 raft_consensus.cc:2229] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.127702 28654 pending_rounds.cc:62] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:17.127718 28655 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.127719 28654 pending_rounds.cc:68] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420033981251584 op_type: NO_OP noop_request { }
I20240503 02:09:17.127749 28654 raft_consensus.cc:2866] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.127769 28654 raft_consensus.cc:2258] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.127761 28661 tablet_replica.cc:330] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.127817 28661 raft_consensus.cc:2229] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.127842 28664 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.127909 28653 ts_tablet_manager.cc:1888] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.127959 28653 log.cc:1196] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d5377c9b1e6c419e8f8a3ed4f0855c06
I20240503 02:09:17.128041 28655 tablet_bootstrap.cc:654] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.128115 28653 ts_tablet_manager.cc:1909] T d5377c9b1e6c419e8f8a3ed4f0855c06 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.128268 28664 tablet_bootstrap.cc:654] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.128430 28655 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.128464 28655 ts_tablet_manager.cc:1367] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.127847 28661 raft_consensus.cc:2258] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.128578 28655 raft_consensus.cc:348] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.128623 28655 raft_consensus.cc:374] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.128638 28655 raft_consensus.cc:729] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.128661 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (table default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]) successfully deleted
I20240503 02:09:17.128672 28655 consensus_queue.cc:260] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.128758 28655 ts_tablet_manager.cc:1398] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.128757 28661 ts_tablet_manager.cc:1875] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.129621 28654 ts_tablet_manager.cc:1875] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.129977 28661 ts_tablet_manager.cc:1888] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.130028 28661 log.cc:1196] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d5377c9b1e6c419e8f8a3ed4f0855c06
I20240503 02:09:17.130192 28661 ts_tablet_manager.cc:1909] T d5377c9b1e6c419e8f8a3ed4f0855c06 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.130705 28654 ts_tablet_manager.cc:1888] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.130756 28654 log.cc:1196] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d5377c9b1e6c419e8f8a3ed4f0855c06
I20240503 02:09:17.130945 28654 ts_tablet_manager.cc:1909] T d5377c9b1e6c419e8f8a3ed4f0855c06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.130928 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (table default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]) successfully deleted
I20240503 02:09:17.131019 28507 consensus_queue.cc:1035] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.131281 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d5377c9b1e6c419e8f8a3ed4f0855c06 (table default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]) successfully deleted
I20240503 02:09:17.131343 28507 consensus_queue.cc:1035] T bcad4e4abb9a47ac9a6c481e91eb1bf9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.131953 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f6a1485d0eca4902b9511e707ab168dd" } modify_external_catalogs: true
I20240503 02:09:17.132050 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f6a1485d0eca4902b9511e707ab168dd" } modify_external_catalogs: true
I20240503 02:09:17.131938 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f2a07194e9874f7f9df96fc754e25779
I20240503 02:09:17.132341 27202 catalog_manager.cc:6554] ReplaceTablet: tablet f2a07194e9874f7f9df96fc754e25779 deleted and replaced by tablet 5924b7e1ff81434885296e6e3ed6757b
I20240503 02:09:17.132506 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a07194e9874f7f9df96fc754e25779 with delete_type TABLET_DATA_DELETED (replaced by tablet 5924b7e1ff81434885296e6e3ed6757b) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.132530 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a07194e9874f7f9df96fc754e25779 with delete_type TABLET_DATA_DELETED (replaced by tablet 5924b7e1ff81434885296e6e3ed6757b) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.132772 28654 tablet_replica.cc:330] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.132830 28654 raft_consensus.cc:2229] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.132859 28654 raft_consensus.cc:2258] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.133422 28654 ts_tablet_manager.cc:1875] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.133692 28664 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.133728 28664 ts_tablet_manager.cc:1367] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.133852 28664 raft_consensus.cc:348] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.133903 28664 raft_consensus.cc:374] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.133920 28664 raft_consensus.cc:729] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.133955 28664 consensus_queue.cc:260] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.134033 28664 ts_tablet_manager.cc:1398] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.134514 28654 ts_tablet_manager.cc:1888] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.134557 28656 tablet_bootstrap.cc:492] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.134605 28656 ts_tablet_manager.cc:1367] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:17.134727 28656 raft_consensus.cc:348] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.134776 28656 raft_consensus.cc:374] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.134794 28656 raft_consensus.cc:729] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.134829 28656 consensus_queue.cc:260] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.134913 28656 ts_tablet_manager.cc:1398] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.134538 28507 consensus_queue.cc:1035] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.135303 27722 raft_consensus.cc:1261] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.135411 27451 raft_consensus.cc:1261] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.136109 28507 consensus_queue.cc:1035] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.136413 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet d002acb8924f4d2194ed1e5f462ebfbc
I20240503 02:09:17.136529 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 051d0dfb8e744d0ca7044f0c03835a90 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.136538 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 051d0dfb8e744d0ca7044f0c03835a90 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.136607 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d0dfb8e744d0ca7044f0c03835a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.136606 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d0dfb8e744d0ca7044f0c03835a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.136828 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a07194e9874f7f9df96fc754e25779 with delete_type TABLET_DATA_DELETED (replaced by tablet 5924b7e1ff81434885296e6e3ed6757b) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.136894 28653 tablet_replica.cc:330] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.136941 28653 raft_consensus.cc:2229] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.136970 28653 raft_consensus.cc:2258] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.137118 28653 ts_tablet_manager.cc:1875] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.137593 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.137655 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.137884 29240 consensus_queue.cc:1035] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.138360 28653 ts_tablet_manager.cc:1888] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.138401 28653 log.cc:1196] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f2a07194e9874f7f9df96fc754e25779
I20240503 02:09:17.138585 28653 ts_tablet_manager.cc:1909] T f2a07194e9874f7f9df96fc754e25779 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.138864 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet d002acb8924f4d2194ed1e5f462ebfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.139138 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 5924b7e1ff81434885296e6e3ed6757b (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.139215 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5924b7e1ff81434885296e6e3ed6757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.139489 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 5924b7e1ff81434885296e6e3ed6757b (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.139472 27195 catalog_manager.cc:5484] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.139559 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (table default.Table_a919652e893b4243a0633f5b0fd85a04 [id=5c0a18e04e06431e8311413c9db7a17a]). Expected version 1 got 0
I20240503 02:09:17.139585 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5924b7e1ff81434885296e6e3ed6757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.139878 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 051d0dfb8e744d0ca7044f0c03835a90 (DEFAULT_TABLE table=default.Table_61db889c4c5a45e793a8bf5976fc5eac [id=ab3d9c3ffb424c999a8b4d897ccc68c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.139968 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 051d0dfb8e744d0ca7044f0c03835a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.140498 28656 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.140705 28767 raft_consensus.cc:482] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.140748 28767 raft_consensus.cc:504] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.140813 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 5924b7e1ff81434885296e6e3ed6757b (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.140846 28767 leader_election.cc:288] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.140846 28656 tablet_bootstrap.cc:654] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.140892 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5924b7e1ff81434885296e6e3ed6757b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.141196 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb958efd36594d5390ae617940aed908" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.141252 27723 raft_consensus.cc:2454] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.141412 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 3297f72c695944debdc8c733280bb86a (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:17.141414 27388 leader_election.cc:302] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.141490 28767 raft_consensus.cc:2781] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.141515 28767 raft_consensus.cc:482] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.141534 28767 raft_consensus.cc:3037] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.141749 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb958efd36594d5390ae617940aed908" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.141803 27583 raft_consensus.cc:2454] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.142131 28767 raft_consensus.cc:504] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.142226 28767 leader_election.cc:288] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.142347 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f272f6f0eb3846598cece6afcc83681f" } modify_external_catalogs: true
I20240503 02:09:17.142400 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f272f6f0eb3846598cece6afcc83681f" } modify_external_catalogs: true
I20240503 02:09:17.142959 27723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb958efd36594d5390ae617940aed908" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.143038 27723 raft_consensus.cc:3037] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.143195 28664 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.143535 28664 tablet_bootstrap.cc:654] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.143589 27723 raft_consensus.cc:2454] T cb958efd36594d5390ae617940aed908 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.143605 28656 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.143641 28656 ts_tablet_manager.cc:1367] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.143757 28656 raft_consensus.cc:348] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.143806 28656 raft_consensus.cc:374] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.143821 28656 raft_consensus.cc:729] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.143854 28656 consensus_queue.cc:260] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.143946 28656 ts_tablet_manager.cc:1398] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.143954 27388 leader_election.cc:302] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.144016 28766 raft_consensus.cc:2781] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.144030 28656 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.144050 28766 raft_consensus.cc:686] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.144080 28766 consensus_queue.cc:237] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.144459 28656 tablet_bootstrap.cc:654] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.144480 27202 master_service.cc:946] ReplaceTablet: received request to replace tablet 63ec489aa11148a4a34b26750a624043 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.144866 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb958efd36594d5390ae617940aed908" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.144920 27583 raft_consensus.cc:3037] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.145025 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 6a0ecd7a67fd4818b25696395a854baf
I20240503 02:09:17.145074 28767 consensus_queue.cc:1035] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.145259 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0ecd7a67fd4818b25696395a854baf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.145351 28661 tablet_replica.cc:330] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.145432 28661 raft_consensus.cc:2229] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.145525 28661 raft_consensus.cc:2258] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.145793 28661 ts_tablet_manager.cc:1875] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.145918 27583 raft_consensus.cc:2454] T cb958efd36594d5390ae617940aed908 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.145925 27202 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 63ec489aa11148a4a34b26750a624043
I20240503 02:09:17.146013 27202 catalog_manager.cc:6554] ReplaceTablet: tablet 63ec489aa11148a4a34b26750a624043 deleted and replaced by tablet 66bbe30f418e4eafaa881e39a45c3dd7
I20240503 02:09:17.146116 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0ecd7a67fd4818b25696395a854baf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.146344 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ec489aa11148a4a34b26750a624043 with delete_type TABLET_DATA_DELETED (replaced by tablet 66bbe30f418e4eafaa881e39a45c3dd7) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.146751 28655 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.146889 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ec489aa11148a4a34b26750a624043 with delete_type TABLET_DATA_DELETED (replaced by tablet 66bbe30f418e4eafaa881e39a45c3dd7) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.147099 28655 tablet_bootstrap.cc:654] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.147294 28661 ts_tablet_manager.cc:1888] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.147338 28661 log.cc:1196] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f2a07194e9874f7f9df96fc754e25779
I20240503 02:09:17.147516 28661 ts_tablet_manager.cc:1909] T f2a07194e9874f7f9df96fc754e25779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.148507 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (DEFAULT_TABLE table=default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.148602 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bbe30f418e4eafaa881e39a45c3dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.148785 28653 tablet_replica.cc:330] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.148837 28653 raft_consensus.cc:2229] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.148813 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (DEFAULT_TABLE table=default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.148838 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a0ecd7a67fd4818b25696395a854baf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.148864 28653 raft_consensus.cc:2258] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.148910 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bbe30f418e4eafaa881e39a45c3dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.148947 28653 ts_tablet_manager.cc:1875] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.150139 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (DEFAULT_TABLE table=default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.150190 28653 ts_tablet_manager.cc:1888] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.150228 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66bbe30f418e4eafaa881e39a45c3dd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.150497 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f2a07194e9874f7f9df96fc754e25779 (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:17.150677 29566 tablet.cc:1722] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.152101 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 63ec489aa11148a4a34b26750a624043 with delete_type TABLET_DATA_DELETED (replaced by tablet 66bbe30f418e4eafaa881e39a45c3dd7) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.134565 28654 log.cc:1196] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f2a07194e9874f7f9df96fc754e25779
I20240503 02:09:17.152354 28654 ts_tablet_manager.cc:1909] T f2a07194e9874f7f9df96fc754e25779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.152683 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f2a07194e9874f7f9df96fc754e25779 (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:17.150231 28653 log.cc:1196] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/d002acb8924f4d2194ed1e5f462ebfbc
I20240503 02:09:17.153034 28653 ts_tablet_manager.cc:1909] T d002acb8924f4d2194ed1e5f462ebfbc P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.153206 28653 tablet_replica.cc:330] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.152495 28654 tablet_replica.cc:330] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.153523 28654 raft_consensus.cc:2229] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.153584 28654 raft_consensus.cc:2258] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.153657 28655 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.153673 28654 ts_tablet_manager.cc:1875] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.153699 28655 ts_tablet_manager.cc:1367] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:17.153779 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 2e29e26955d441ef9f7ca908da29ca49 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.153839 28655 raft_consensus.cc:348] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.153895 28655 raft_consensus.cc:374] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.153915 28655 raft_consensus.cc:729] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.153945 28655 consensus_queue.cc:260] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.154016 28655 ts_tablet_manager.cc:1398] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.154062 28655 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.154069 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet d002acb8924f4d2194ed1e5f462ebfbc (table default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]) successfully deleted
I20240503 02:09:17.154428 28655 tablet_bootstrap.cc:654] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.154467 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18] alter complete (version 1)
I20240503 02:09:17.153427 29241 consensus_queue.cc:1035] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:09:17.154928 28710 tablet.cc:1722] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.155109 28654 ts_tablet_manager.cc:1888] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.155154 28654 log.cc:1196] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/d002acb8924f4d2194ed1e5f462ebfbc
I20240503 02:09:17.155443 28654 ts_tablet_manager.cc:1909] T d002acb8924f4d2194ed1e5f462ebfbc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.155468 29558 tablet.cc:1722] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.155694 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet d002acb8924f4d2194ed1e5f462ebfbc (table default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]) successfully deleted
I20240503 02:09:17.155841 28654 tablet_replica.cc:330] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.155929 28654 raft_consensus.cc:2229] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.155965 28654 raft_consensus.cc:2258] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.156076 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f2a07194e9874f7f9df96fc754e25779 (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:17.156158 28654 ts_tablet_manager.cc:1875] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.156766 28661 tablet_replica.cc:330] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.156910 28661 raft_consensus.cc:2229] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.156948 28661 raft_consensus.cc:2258] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.157050 28661 ts_tablet_manager.cc:1875] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.157092 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 3900c1017d114230bc93d16ece225e3f from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.157214 27201 catalog_manager.cc:5484] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.154058 28653 raft_consensus.cc:2229] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.158145 28653 raft_consensus.cc:2258] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.158428 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2e29e26955d441ef9f7ca908da29ca49
I20240503 02:09:17.158532 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 2e29e26955d441ef9f7ca908da29ca49 deleted and replaced by tablet abaaf307b73e46b6a5548a0acba68ba9
I20240503 02:09:17.158622 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 with delete_type TABLET_DATA_DELETED (replaced by tablet abaaf307b73e46b6a5548a0acba68ba9) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.158675 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 with delete_type TABLET_DATA_DELETED (replaced by tablet abaaf307b73e46b6a5548a0acba68ba9) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.158824 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e29e26955d441ef9f7ca908da29ca49 with delete_type TABLET_DATA_DELETED (replaced by tablet abaaf307b73e46b6a5548a0acba68ba9) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.159183 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3900c1017d114230bc93d16ece225e3f
I20240503 02:09:17.159250 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 3900c1017d114230bc93d16ece225e3f deleted and replaced by tablet 2081f9c133424955a57f3b9362c8fbd8
I20240503 02:09:17.159328 28653 ts_tablet_manager.cc:1875] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.159571 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3900c1017d114230bc93d16ece225e3f with delete_type TABLET_DATA_DELETED (replaced by tablet 2081f9c133424955a57f3b9362c8fbd8) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.159900 28661 ts_tablet_manager.cc:1888] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.159978 28661 log.cc:1196] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/d002acb8924f4d2194ed1e5f462ebfbc
I20240503 02:09:17.160027 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 3900c1017d114230bc93d16ece225e3f with delete_type TABLET_DATA_DELETED (replaced by tablet 2081f9c133424955a57f3b9362c8fbd8) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.160075 28654 ts_tablet_manager.cc:1888] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.160120 28654 log.cc:1196] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/63ec489aa11148a4a34b26750a624043
I20240503 02:09:17.160224 28661 ts_tablet_manager.cc:1909] T d002acb8924f4d2194ed1e5f462ebfbc P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.160301 28654 ts_tablet_manager.cc:1909] T 63ec489aa11148a4a34b26750a624043 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.160537 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet d002acb8924f4d2194ed1e5f462ebfbc (table default.Table_f6a1485d0eca4902b9511e707ab168dd [id=8bb62e20da33409f92b03215e253fe58]) successfully deleted
I20240503 02:09:17.160538 28661 tablet_replica.cc:330] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.160595 28661 raft_consensus.cc:2229] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.160629 28661 raft_consensus.cc:2258] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.160670 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 63ec489aa11148a4a34b26750a624043 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:17.160880 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 3900c1017d114230bc93d16ece225e3f with delete_type TABLET_DATA_DELETED (replaced by tablet 2081f9c133424955a57f3b9362c8fbd8) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.160929 28661 ts_tablet_manager.cc:1875] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.161365 27583 raft_consensus.cc:1261] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.161389 28409 consensus_queue.cc:1035] T 1f6d82db9bc94f95ad0e602a43bcdc87 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.161581 27718 raft_consensus.cc:1261] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.161970 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 (DEFAULT_TABLE table=default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.161993 27560 tablet_service.cc:1467] Processing CreateTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.162061 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2081f9c133424955a57f3b9362c8fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.162119 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 (DEFAULT_TABLE table=default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.162192 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2081f9c133424955a57f3b9362c8fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.162216 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaaf307b73e46b6a5548a0acba68ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.162458 28409 consensus_queue.cc:1035] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.163154 27702 tablet_service.cc:1467] Processing CreateTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.163234 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaaf307b73e46b6a5548a0acba68ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.163221 28409 consensus_queue.cc:1035] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.163935 28656 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.163991 28656 ts_tablet_manager.cc:1367] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.000s
I20240503 02:09:17.164108 28656 raft_consensus.cc:348] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.164156 28656 raft_consensus.cc:374] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.164173 28656 raft_consensus.cc:729] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.164206 28656 consensus_queue.cc:260] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.164284 28656 ts_tablet_manager.cc:1398] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.164331 28656 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.164393 27425 tablet_service.cc:1467] Processing CreateTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.164436 27450 raft_consensus.cc:1261] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.164475 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaaf307b73e46b6a5548a0acba68ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.164525 27718 raft_consensus.cc:1261] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.164664 28656 tablet_bootstrap.cc:654] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.164829 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 (DEFAULT_TABLE table=default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.164907 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2081f9c133424955a57f3b9362c8fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.165478 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:17.165745 29241 consensus_queue.cc:1035] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.166594 29241 consensus_queue.cc:1035] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.166762 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]). Expected version 1 got 0
I20240503 02:09:17.167268 28655 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.167316 28655 ts_tablet_manager.cc:1367] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:17.167342 28661 ts_tablet_manager.cc:1888] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.167393 28661 log.cc:1196] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/6a0ecd7a67fd4818b25696395a854baf
I20240503 02:09:17.167428 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (table default.Table_a919652e893b4243a0633f5b0fd85a04 [id=5c0a18e04e06431e8311413c9db7a17a]). Expected version 1 got 0
I20240503 02:09:17.167543 28655 raft_consensus.cc:348] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.167574 28661 ts_tablet_manager.cc:1909] T 6a0ecd7a67fd4818b25696395a854baf P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.167599 28655 raft_consensus.cc:374] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.167618 28655 raft_consensus.cc:729] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.167657 28655 consensus_queue.cc:260] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.167691 28654 tablet_replica.cc:330] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.167745 28654 raft_consensus.cc:2229] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.167764 28655 ts_tablet_manager.cc:1398] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.167784 28654 pending_rounds.cc:62] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:17.167801 28654 pending_rounds.cc:68] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420035519373312 op_type: NO_OP noop_request { }
I20240503 02:09:17.167822 28655 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.167832 28654 raft_consensus.cc:2866] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.167853 28654 raft_consensus.cc:2258] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.168092 28654 ts_tablet_manager.cc:1875] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.168133 28655 tablet_bootstrap.cc:654] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.168129 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet fcc99e566bc24ad6a98d20dc61bcfa6d (table default.Table_a919652e893b4243a0633f5b0fd85a04 [id=5c0a18e04e06431e8311413c9db7a17a]). Expected version 1 got 0
I20240503 02:09:17.169713 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 6a0ecd7a67fd4818b25696395a854baf (table default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]) successfully deleted
I20240503 02:09:17.169816 28661 tablet_replica.cc:330] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.169821 28653 ts_tablet_manager.cc:1888] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.169863 28653 log.cc:1196] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/6a0ecd7a67fd4818b25696395a854baf
I20240503 02:09:17.169869 28654 ts_tablet_manager.cc:1888] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.169898 28661 raft_consensus.cc:2229] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.169934 28654 log.cc:1196] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/6a0ecd7a67fd4818b25696395a854baf
I20240503 02:09:17.170028 28653 ts_tablet_manager.cc:1909] T 6a0ecd7a67fd4818b25696395a854baf P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.170027 28661 raft_consensus.cc:2258] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.170159 28654 ts_tablet_manager.cc:1909] T 6a0ecd7a67fd4818b25696395a854baf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.170228 28653 tablet_replica.cc:330] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.170238 28661 ts_tablet_manager.cc:1875] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.170275 28653 raft_consensus.cc:2229] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.170295 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 6a0ecd7a67fd4818b25696395a854baf (table default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]) successfully deleted
I20240503 02:09:17.170310 28653 raft_consensus.cc:2258] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.170486 28653 ts_tablet_manager.cc:1875] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.170625 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 6a0ecd7a67fd4818b25696395a854baf (table default.Table_f272f6f0eb3846598cece6afcc83681f [id=a035d8da3d524914a202eb6c91735666]) successfully deleted
I20240503 02:09:17.170670 29241 raft_consensus.cc:482] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.170707 29241 raft_consensus.cc:504] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.170840 29241 leader_election.cc:288] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.170982 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c14768b631a423084b6f43203a8832e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.171002 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c14768b631a423084b6f43203a8832e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.171036 27449 raft_consensus.cc:2454] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.171063 27719 raft_consensus.cc:2454] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.171140 27524 leader_election.cc:302] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.171211 29241 raft_consensus.cc:2781] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.171233 29241 raft_consensus.cc:482] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.171249 29241 raft_consensus.cc:3037] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.171715 28661 ts_tablet_manager.cc:1888] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.171756 28661 log.cc:1196] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/63ec489aa11148a4a34b26750a624043
I20240503 02:09:17.171759 28653 ts_tablet_manager.cc:1888] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.171801 28653 log.cc:1196] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/63ec489aa11148a4a34b26750a624043
I20240503 02:09:17.171795 29241 raft_consensus.cc:504] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.171905 29241 leader_election.cc:288] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.171973 28653 ts_tablet_manager.cc:1909] T 63ec489aa11148a4a34b26750a624043 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.172032 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c14768b631a423084b6f43203a8832e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.172055 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c14768b631a423084b6f43203a8832e" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.172082 27449 raft_consensus.cc:3037] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.172106 27719 raft_consensus.cc:3037] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.172333 28653 tablet_replica.cc:330] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.172380 28653 raft_consensus.cc:2229] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.172405 28653 raft_consensus.cc:2258] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.172488 28661 ts_tablet_manager.cc:1909] T 63ec489aa11148a4a34b26750a624043 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.172672 27449 raft_consensus.cc:2454] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.172672 27719 raft_consensus.cc:2454] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.172745 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 63ec489aa11148a4a34b26750a624043 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:17.172879 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 63ec489aa11148a4a34b26750a624043 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]) successfully deleted
I20240503 02:09:17.172950 27524 leader_election.cc:302] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.173009 29241 raft_consensus.cc:2781] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.173039 29241 raft_consensus.cc:686] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.173072 29241 consensus_queue.cc:237] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.173449 28661 tablet_replica.cc:330] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.173506 28661 raft_consensus.cc:2229] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.173547 28661 pending_rounds.cc:62] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:17.173563 28661 pending_rounds.cc:68] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420034907545600 op_type: NO_OP noop_request { }
I20240503 02:09:17.173594 28661 raft_consensus.cc:2866] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.173614 28661 raft_consensus.cc:2258] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.173921 28653 ts_tablet_manager.cc:1875] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.174417 28655 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.174458 28655 ts_tablet_manager.cc:1367] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:17.174587 28655 raft_consensus.cc:348] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.174650 28655 raft_consensus.cc:374] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.174679 28655 raft_consensus.cc:729] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.174712 28655 consensus_queue.cc:260] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.174728 28710 tablet.cc:1722] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.174810 28655 ts_tablet_manager.cc:1398] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.174858 28655 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.175206 28655 tablet_bootstrap.cc:654] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.175690 28653 ts_tablet_manager.cc:1888] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.175734 28653 log.cc:1196] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2e29e26955d441ef9f7ca908da29ca49
I20240503 02:09:17.175876 28653 ts_tablet_manager.cc:1909] T 2e29e26955d441ef9f7ca908da29ca49 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.170698 28654 tablet_replica.cc:330] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.175947 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.176013 28654 raft_consensus.cc:2229] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.176044 28654 raft_consensus.cc:2258] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:17.176039 27202 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 66bbe30f418e4eafaa881e39a45c3dd7 consensus state has no leader
I20240503 02:09:17.176208 28654 ts_tablet_manager.cc:1875] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.176283 28661 ts_tablet_manager.cc:1875] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.176530 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2e29e26955d441ef9f7ca908da29ca49 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.176733 28653 tablet_replica.cc:330] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.176787 28653 raft_consensus.cc:2229] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.176817 28653 raft_consensus.cc:2258] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.176976 28653 ts_tablet_manager.cc:1875] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.177078 27202 catalog_manager.cc:5484] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.177440 28654 ts_tablet_manager.cc:1888] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.177480 28654 log.cc:1196] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2e29e26955d441ef9f7ca908da29ca49
I20240503 02:09:17.177613 28654 ts_tablet_manager.cc:1909] T 2e29e26955d441ef9f7ca908da29ca49 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.178166 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2e29e26955d441ef9f7ca908da29ca49 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.178246 28654 tablet_replica.cc:330] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.178295 28654 raft_consensus.cc:2229] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.178344 28654 raft_consensus.cc:2258] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.178431 28654 ts_tablet_manager.cc:1875] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.178548 29566 tablet.cc:1722] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.178810 28653 ts_tablet_manager.cc:1888] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.178864 28653 log.cc:1196] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3900c1017d114230bc93d16ece225e3f
I20240503 02:09:17.179045 28653 ts_tablet_manager.cc:1909] T 3900c1017d114230bc93d16ece225e3f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.179366 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
I20240503 02:09:17.179527 28654 ts_tablet_manager.cc:1888] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.179569 28654 log.cc:1196] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3900c1017d114230bc93d16ece225e3f
I20240503 02:09:17.179641 28661 ts_tablet_manager.cc:1888] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.179684 28661 log.cc:1196] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2e29e26955d441ef9f7ca908da29ca49
I20240503 02:09:17.179745 28654 ts_tablet_manager.cc:1909] T 3900c1017d114230bc93d16ece225e3f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.179857 28661 ts_tablet_manager.cc:1909] T 2e29e26955d441ef9f7ca908da29ca49 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.180009 28656 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.180049 28656 ts_tablet_manager.cc:1367] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.000s
I20240503 02:09:17.180162 28656 raft_consensus.cc:348] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.180224 28656 raft_consensus.cc:374] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.180248 28656 raft_consensus.cc:729] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.180251 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2e29e26955d441ef9f7ca908da29ca49 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.180280 28656 consensus_queue.cc:260] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.180316 28661 tablet_replica.cc:330] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.180356 28656 ts_tablet_manager.cc:1398] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.180366 28661 raft_consensus.cc:2229] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.180402 28661 raft_consensus.cc:2258] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.180413 28656 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.180490 28661 ts_tablet_manager.cc:1875] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.180724 28656 tablet_bootstrap.cc:654] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.181412 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.181444 28664 tablet_bootstrap.cc:492] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.181489 28664 ts_tablet_manager.cc:1367] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.000s
I20240503 02:09:17.181617 28664 raft_consensus.cc:348] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.181668 28664 raft_consensus.cc:374] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.181684 28664 raft_consensus.cc:729] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.181720 28664 consensus_queue.cc:260] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.181795 28664 ts_tablet_manager.cc:1398] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.181859 28664 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.182071 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
I20240503 02:09:17.182124 28661 ts_tablet_manager.cc:1888] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.182175 28664 tablet_bootstrap.cc:654] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.182163 28661 log.cc:1196] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3900c1017d114230bc93d16ece225e3f
I20240503 02:09:17.183058 28655 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.183094 28655 ts_tablet_manager.cc:1367] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:17.183094 28661 ts_tablet_manager.cc:1909] T 3900c1017d114230bc93d16ece225e3f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.183219 28655 raft_consensus.cc:348] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.183270 28655 raft_consensus.cc:374] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.183286 28655 raft_consensus.cc:729] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.183311 28655 consensus_queue.cc:260] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.183388 28655 ts_tablet_manager.cc:1398] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.183480 28655 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.183586 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3900c1017d114230bc93d16ece225e3f (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
I20240503 02:09:17.183807 28655 tablet_bootstrap.cc:654] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.184355 28655 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.184391 28655 ts_tablet_manager.cc:1367] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.184505 28655 raft_consensus.cc:348] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.184556 28655 raft_consensus.cc:374] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.184573 28655 raft_consensus.cc:729] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.184607 28655 consensus_queue.cc:260] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.184705 28655 ts_tablet_manager.cc:1398] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.184911 28664 tablet_bootstrap.cc:492] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.184945 28664 ts_tablet_manager.cc:1367] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.185063 28664 raft_consensus.cc:348] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.185111 28664 raft_consensus.cc:374] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.185127 28664 raft_consensus.cc:729] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.185159 28664 consensus_queue.cc:260] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.185228 28664 ts_tablet_manager.cc:1398] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.185281 28664 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.185644 28664 tablet_bootstrap.cc:654] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.186182 28664 tablet_bootstrap.cc:492] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.186224 28664 ts_tablet_manager.cc:1367] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.186354 28664 raft_consensus.cc:348] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.186421 28664 raft_consensus.cc:374] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.186440 28664 raft_consensus.cc:729] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.186472 28664 consensus_queue.cc:260] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.186548 28664 ts_tablet_manager.cc:1398] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.186554 29558 tablet.cc:1722] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.186614 28664 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.186640 28656 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.186672 28656 ts_tablet_manager.cc:1367] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.186790 28656 raft_consensus.cc:348] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.186836 28656 raft_consensus.cc:374] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.186853 28656 raft_consensus.cc:729] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.186889 28656 consensus_queue.cc:260] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.186926 28664 tablet_bootstrap.cc:654] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.186968 28656 ts_tablet_manager.cc:1398] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.187029 28656 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.187363 28656 tablet_bootstrap.cc:654] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.187449 28774 raft_consensus.cc:482] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.187500 28774 raft_consensus.cc:504] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.187582 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.187638 28774 leader_election.cc:288] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.187968 28774 raft_consensus.cc:482] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.188017 28774 raft_consensus.cc:504] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.188108 28774 leader_election.cc:288] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.188320 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2983e9a39aed45f29aaaca9c5c010bb8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.188388 27582 raft_consensus.cc:2454] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.188320 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec4ddee00974610917b834da66647d3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.188468 27583 raft_consensus.cc:2454] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.188767 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.188800 27656 leader_election.cc:302] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
W20240503 02:09:17.188828 27195 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 2081f9c133424955a57f3b9362c8fbd8 consensus state has no leader
I20240503 02:09:17.188874 27656 leader_election.cc:302] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.188933 28774 raft_consensus.cc:2781] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.188930 28740 raft_consensus.cc:2781] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.189000 28740 raft_consensus.cc:482] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.189019 28740 raft_consensus.cc:3037] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.189201 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.188964 28774 raft_consensus.cc:482] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.189489 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2983e9a39aed45f29aaaca9c5c010bb8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.189530 28774 raft_consensus.cc:3037] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.189549 27451 raft_consensus.cc:2454] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.189486 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec4ddee00974610917b834da66647d3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.189628 27449 raft_consensus.cc:2454] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.189805 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.190057 28774 raft_consensus.cc:504] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.190148 28774 leader_election.cc:288] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.190443 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec4ddee00974610917b834da66647d3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.190496 27449 raft_consensus.cc:3037] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.190554 28664 tablet_bootstrap.cc:492] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.190591 28664 ts_tablet_manager.cc:1367] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.190596 28656 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.190629 28656 ts_tablet_manager.cc:1367] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.190707 28664 raft_consensus.cc:348] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.190755 28664 raft_consensus.cc:374] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.190773 28664 raft_consensus.cc:729] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.190786 28740 raft_consensus.cc:504] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.190806 28664 consensus_queue.cc:260] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.190866 28740 leader_election.cc:288] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.190850 28656 raft_consensus.cc:348] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.190903 28656 raft_consensus.cc:374] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.190908 28664 ts_tablet_manager.cc:1398] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.190922 28656 raft_consensus.cc:729] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.190966 28664 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.190961 28656 consensus_queue.cc:260] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.191064 28656 ts_tablet_manager.cc:1398] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.191079 27449 raft_consensus.cc:2454] T cec4ddee00974610917b834da66647d3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.191128 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cec4ddee00974610917b834da66647d3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.191165 27583 raft_consensus.cc:3037] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.191229 27659 leader_election.cc:302] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.191313 28664 tablet_bootstrap.cc:654] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.191323 28774 raft_consensus.cc:2781] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.191355 28774 raft_consensus.cc:686] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.191545 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2983e9a39aed45f29aaaca9c5c010bb8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.191617 27449 raft_consensus.cc:3037] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.191790 27583 raft_consensus.cc:2454] T cec4ddee00974610917b834da66647d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.191864 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2983e9a39aed45f29aaaca9c5c010bb8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.191905 27582 raft_consensus.cc:3037] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.192467 27582 raft_consensus.cc:2454] T 2983e9a39aed45f29aaaca9c5c010bb8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.192750 27449 raft_consensus.cc:2454] T 2983e9a39aed45f29aaaca9c5c010bb8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.193374 27656 leader_election.cc:302] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.191753 28740 raft_consensus.cc:482] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.193436 28740 raft_consensus.cc:504] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.193521 28740 leader_election.cc:288] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.193548 28740 raft_consensus.cc:2781] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.193578 28740 raft_consensus.cc:686] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.193610 28740 consensus_queue.cc:237] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.193990 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3650180caf546878779d1c89950b2c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.194046 27449 raft_consensus.cc:2454] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.194308 28767 consensus_queue.cc:1035] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.194731 29566 tablet.cc:1722] T fcc99e566bc24ad6a98d20dc61bcfa6d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.194775 29181 mvcc.cc:205] Tried to move back new op lower bound from 7023420035738492928 to 7023420035579047936. Current Snapshot: MvccSnapshot[applied={T|T < 7023420035738492928}]
I20240503 02:09:17.194833 29558 tablet.cc:1722] T fcc99e566bc24ad6a98d20dc61bcfa6d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.191442 28774 consensus_queue.cc:237] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.196313 28710 tablet.cc:1722] T fcc99e566bc24ad6a98d20dc61bcfa6d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.196377 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3650180caf546878779d1c89950b2c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.196429 27582 raft_consensus.cc:2454] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.196903 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.196970 27659 leader_election.cc:302] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.196965 27202 catalog_manager.cc:5484] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.197057 29643 raft_consensus.cc:2781] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.197084 29643 raft_consensus.cc:482] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.197100 29643 raft_consensus.cc:3037] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.197659 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_a919652e893b4243a0633f5b0fd85a04 [id=5c0a18e04e06431e8311413c9db7a17a] alter complete (version 1)
I20240503 02:09:17.197919 29643 raft_consensus.cc:504] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.198019 29643 leader_election.cc:288] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.199261 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3650180caf546878779d1c89950b2c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.199316 27582 raft_consensus.cc:3037] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.199867 27582 raft_consensus.cc:2454] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.199823 27195 catalog_manager.cc:5484] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.200361 27656 leader_election.cc:302] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.200773 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3650180caf546878779d1c89950b2c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.200842 27450 raft_consensus.cc:3037] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.201475 28774 raft_consensus.cc:2781] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.201529 28774 raft_consensus.cc:686] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.201546 27450 raft_consensus.cc:2454] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.201568 28774 consensus_queue.cc:237] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.201607 28507 raft_consensus.cc:482] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.201650 28507 raft_consensus.cc:504] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.201738 28507 leader_election.cc:288] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.202026 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43948f8039d948efbf1dcde10e012be4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.202082 27450 raft_consensus.cc:2454] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.202181 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43948f8039d948efbf1dcde10e012be4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.202230 27722 raft_consensus.cc:2454] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.202387 27523 leader_election.cc:302] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.203557 28507 raft_consensus.cc:2781] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.203594 28507 raft_consensus.cc:482] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.203611 28507 raft_consensus.cc:3037] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.203624 27202 catalog_manager.cc:5484] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.203711 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.204205 28507 raft_consensus.cc:504] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.204262 28664 tablet_bootstrap.cc:492] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.204293 28664 ts_tablet_manager.cc:1367] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:17.204327 28507 leader_election.cc:288] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.204450 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43948f8039d948efbf1dcde10e012be4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.204464 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43948f8039d948efbf1dcde10e012be4" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.204501 27450 raft_consensus.cc:3037] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.204515 27718 raft_consensus.cc:3037] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.205067 27718 raft_consensus.cc:2454] T 43948f8039d948efbf1dcde10e012be4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.205075 27450 raft_consensus.cc:2454] T 43948f8039d948efbf1dcde10e012be4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.205209 27523 leader_election.cc:302] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.205370 28409 raft_consensus.cc:482] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.205379 28664 raft_consensus.cc:348] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.205413 28409 raft_consensus.cc:504] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.205439 28664 raft_consensus.cc:374] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.205466 28664 raft_consensus.cc:729] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.205497 28409 leader_election.cc:288] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.205504 28664 consensus_queue.cc:260] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.205592 28664 ts_tablet_manager.cc:1398] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.206761 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7116932a247407196f5c17ddc1b4e9f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.206789 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7116932a247407196f5c17ddc1b4e9f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.206818 27718 raft_consensus.cc:2454] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.206831 27583 raft_consensus.cc:2454] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.207080 29241 raft_consensus.cc:2781] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.207129 29241 raft_consensus.cc:686] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.207142 27386 leader_election.cc:302] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.207165 29241 consensus_queue.cc:237] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.207228 28767 raft_consensus.cc:2781] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.207271 28767 raft_consensus.cc:482] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.207217 29550 consensus_queue.cc:1035] T 036d2832bef249b3b350c8916297ffbb P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.207290 28767 raft_consensus.cc:3037] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.207320 29241 raft_consensus.cc:482] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.207350 29241 raft_consensus.cc:504] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.207417 29241 leader_election.cc:288] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.207901 28767 raft_consensus.cc:504] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.207994 28767 leader_election.cc:288] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.208158 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7116932a247407196f5c17ddc1b4e9f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.208214 27582 raft_consensus.cc:3037] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.208269 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8f3809378f444386cdd868e6db4c04" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.208326 27718 raft_consensus.cc:2454] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.208513 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7116932a247407196f5c17ddc1b4e9f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.208528 27523 leader_election.cc:302] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.208560 27718 raft_consensus.cc:3037] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.208570 27202 catalog_manager.cc:5484] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.208603 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8f3809378f444386cdd868e6db4c04" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.208649 27450 raft_consensus.cc:2454] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.208763 27582 raft_consensus.cc:2454] T a7116932a247407196f5c17ddc1b4e9f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.209007 27386 leader_election.cc:302] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.209072 28767 raft_consensus.cc:2781] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.209075 27718 raft_consensus.cc:2454] T a7116932a247407196f5c17ddc1b4e9f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.209090 29241 raft_consensus.cc:2781] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.209103 28767 raft_consensus.cc:686] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.209117 29241 raft_consensus.cc:482] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.209136 29241 raft_consensus.cc:3037] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.209138 28767 consensus_queue.cc:237] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.209728 29241 raft_consensus.cc:504] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.209839 29241 leader_election.cc:288] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.209972 27202 catalog_manager.cc:5484] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.210314 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8f3809378f444386cdd868e6db4c04" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.210371 27718 raft_consensus.cc:3037] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.210857 27718 raft_consensus.cc:2454] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.211053 27523 leader_election.cc:302] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.211135 29241 raft_consensus.cc:2781] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.211186 29241 raft_consensus.cc:686] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.211200 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a8f3809378f444386cdd868e6db4c04" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.211220 29241 consensus_queue.cc:237] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.211273 27450 raft_consensus.cc:3037] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.211815 27450 raft_consensus.cc:2454] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.212323 27195 catalog_manager.cc:5484] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.217990 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3826661f099d46e0bacb75ec4454ee94" } modify_external_catalogs: true
I20240503 02:09:17.218068 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3826661f099d46e0bacb75ec4454ee94" } modify_external_catalogs: true
I20240503 02:09:17.218899 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4782d97b11e04c0ba788b9af68ab8802
I20240503 02:09:17.219091 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.219132 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.219197 28653 tablet_replica.cc:330] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.219223 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 4782d97b11e04c0ba788b9af68ab8802 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.219259 28653 raft_consensus.cc:2229] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.219286 28654 tablet_replica.cc:330] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.219308 28653 raft_consensus.cc:2258] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.219337 28654 raft_consensus.cc:2229] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.219399 28654 raft_consensus.cc:2258] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.219442 28653 ts_tablet_manager.cc:1875] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.219485 28654 ts_tablet_manager.cc:1875] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.220736 28653 ts_tablet_manager.cc:1888] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.220736 28654 ts_tablet_manager.cc:1888] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.220779 28653 log.cc:1196] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4782d97b11e04c0ba788b9af68ab8802
I20240503 02:09:17.220779 28654 log.cc:1196] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4782d97b11e04c0ba788b9af68ab8802
I20240503 02:09:17.220959 28653 ts_tablet_manager.cc:1909] T 4782d97b11e04c0ba788b9af68ab8802 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.220965 28654 ts_tablet_manager.cc:1909] T 4782d97b11e04c0ba788b9af68ab8802 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.221413 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_3826661f099d46e0bacb75ec4454ee94 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:17.221767 28661 tablet_replica.cc:330] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.221757 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_3826661f099d46e0bacb75ec4454ee94 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:17.221894 28661 raft_consensus.cc:2229] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.221935 28661 raft_consensus.cc:2258] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.222059 28661 ts_tablet_manager.cc:1875] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.223002 28661 ts_tablet_manager.cc:1888] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:17.223049 28661 log.cc:1196] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4782d97b11e04c0ba788b9af68ab8802
I20240503 02:09:17.223219 28661 ts_tablet_manager.cc:1909] T 4782d97b11e04c0ba788b9af68ab8802 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.223515 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4782d97b11e04c0ba788b9af68ab8802 (table default.Table_3826661f099d46e0bacb75ec4454ee94 [id=87148f42b2134ac5accf54e9b542661a]) successfully deleted
I20240503 02:09:17.230842 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.230935 27202 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
W20240503 02:09:17.230966 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.233145 28767 consensus_queue.cc:1035] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.233417 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.233425 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.233518 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25063c03d7dd4e7a87b7e8bd315684a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.233527 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25063c03d7dd4e7a87b7e8bd315684a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.233577 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.233654 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25063c03d7dd4e7a87b7e8bd315684a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.234313 28664 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.234714 28664 tablet_bootstrap.cc:654] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.234799 28656 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.234799 28655 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.235208 28656 tablet_bootstrap.cc:654] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.235241 28655 tablet_bootstrap.cc:654] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.235740 28656 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.235785 28656 ts_tablet_manager.cc:1367] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.235790 28655 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.235831 28655 ts_tablet_manager.cc:1367] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.235937 28656 raft_consensus.cc:348] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.236003 28656 raft_consensus.cc:374] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.235981 28655 raft_consensus.cc:348] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.236027 28656 raft_consensus.cc:729] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.236032 28655 raft_consensus.cc:374] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.236049 28655 raft_consensus.cc:729] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.236071 28656 consensus_queue.cc:260] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.236089 28655 consensus_queue.cc:260] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.236169 28655 ts_tablet_manager.cc:1398] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.236176 28656 ts_tablet_manager.cc:1398] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.238492 28774 raft_consensus.cc:482] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.238541 28774 raft_consensus.cc:504] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.238631 28774 leader_election.cc:288] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.238729 29551 consensus_queue.cc:1035] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.238835 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0182ed8465eb40238a7efc2850f18173" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.238880 27451 raft_consensus.cc:2454] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.239260 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0182ed8465eb40238a7efc2850f18173" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.239343 27582 raft_consensus.cc:2454] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.239585 27202 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.240362 27656 leader_election.cc:302] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.240458 28774 raft_consensus.cc:2781] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.240483 28774 raft_consensus.cc:482] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.240499 28774 raft_consensus.cc:3037] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.240839 28664 tablet_bootstrap.cc:492] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.240875 28664 ts_tablet_manager.cc:1367] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:17.241104 28774 raft_consensus.cc:504] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.241123 28664 raft_consensus.cc:348] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.241163 28664 raft_consensus.cc:374] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.241180 28664 raft_consensus.cc:729] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.241210 28774 leader_election.cc:288] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.241223 28664 consensus_queue.cc:260] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.241312 28664 ts_tablet_manager.cc:1398] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.241367 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0182ed8465eb40238a7efc2850f18173" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.241426 27582 raft_consensus.cc:3037] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.241729 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0182ed8465eb40238a7efc2850f18173" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.241782 27451 raft_consensus.cc:3037] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.242018 27582 raft_consensus.cc:2454] T 0182ed8465eb40238a7efc2850f18173 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.242269 27656 leader_election.cc:302] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.242383 28774 raft_consensus.cc:2781] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.242420 28774 raft_consensus.cc:686] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.242415 27451 raft_consensus.cc:2454] T 0182ed8465eb40238a7efc2850f18173 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.242455 28774 consensus_queue.cc:237] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.243899 27195 catalog_manager.cc:5484] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.245282 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4926b749765c4232b2834d5419cc1ab1" } modify_external_catalogs: true
I20240503 02:09:17.245360 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4926b749765c4232b2834d5419cc1ab1" } modify_external_catalogs: true
I20240503 02:09:17.246229 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a710cef4241b47d788dced187487c7cd
I20240503 02:09:17.246439 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet a710cef4241b47d788dced187487c7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.246471 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet a710cef4241b47d788dced187487c7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.246519 28653 tablet_replica.cc:330] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.246562 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet a710cef4241b47d788dced187487c7cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.246588 28653 raft_consensus.cc:2229] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.246620 28653 raft_consensus.cc:2258] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.246618 28654 tablet_replica.cc:330] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.246668 28654 raft_consensus.cc:2229] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.246696 28654 raft_consensus.cc:2258] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.246810 28653 ts_tablet_manager.cc:1875] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.246953 28654 ts_tablet_manager.cc:1875] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.248081 28653 ts_tablet_manager.cc:1888] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.248124 28653 log.cc:1196] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a710cef4241b47d788dced187487c7cd
I20240503 02:09:17.248193 28654 ts_tablet_manager.cc:1888] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.248229 28654 log.cc:1196] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a710cef4241b47d788dced187487c7cd
I20240503 02:09:17.248299 28653 ts_tablet_manager.cc:1909] T a710cef4241b47d788dced187487c7cd P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.248392 28654 ts_tablet_manager.cc:1909] T a710cef4241b47d788dced187487c7cd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.248656 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a710cef4241b47d788dced187487c7cd (table default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]) successfully deleted
I20240503 02:09:17.248677 28661 tablet_replica.cc:330] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.248736 28661 raft_consensus.cc:2229] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.248790 28661 raft_consensus.cc:2258] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.249018 28661 ts_tablet_manager.cc:1875] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.249070 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a710cef4241b47d788dced187487c7cd (table default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]) successfully deleted
I20240503 02:09:17.250386 28661 ts_tablet_manager.cc:1888] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.250429 28661 log.cc:1196] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a710cef4241b47d788dced187487c7cd
I20240503 02:09:17.250597 28661 ts_tablet_manager.cc:1909] T a710cef4241b47d788dced187487c7cd P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.250919 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a710cef4241b47d788dced187487c7cd (table default.Table_4926b749765c4232b2834d5419cc1ab1 [id=d19bbf727c444f1f83658cb52ff9358d]) successfully deleted
I20240503 02:09:17.250972 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e7d8d66c8a284fa69d3f8732a45da2be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.251088 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7d8d66c8a284fa69d3f8732a45da2be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.253017 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 58e3e21135574e9bba4f1c6896ee545c (DEFAULT_TABLE table=default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.253038 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 58e3e21135574e9bba4f1c6896ee545c (DEFAULT_TABLE table=default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.253121 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e3e21135574e9bba4f1c6896ee545c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.253134 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e3e21135574e9bba4f1c6896ee545c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.253211 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 58e3e21135574e9bba4f1c6896ee545c (DEFAULT_TABLE table=default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.253304 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e3e21135574e9bba4f1c6896ee545c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.254244 28655 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.254534 28656 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.254611 28655 tablet_bootstrap.cc:654] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.254850 28664 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.254900 28656 tablet_bootstrap.cc:654] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.255033 28655 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.255067 28655 ts_tablet_manager.cc:1367] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.255160 28664 tablet_bootstrap.cc:654] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.255190 28655 raft_consensus.cc:348] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.255246 28655 raft_consensus.cc:374] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.255286 28655 raft_consensus.cc:729] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.255321 28655 consensus_queue.cc:260] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.255430 28655 ts_tablet_manager.cc:1398] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.256583 28664 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.256621 28664 ts_tablet_manager.cc:1367] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:17.256750 28664 raft_consensus.cc:348] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.256803 28664 raft_consensus.cc:374] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.256817 28664 raft_consensus.cc:729] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.256851 28664 consensus_queue.cc:260] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.256932 28664 ts_tablet_manager.cc:1398] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.257472 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a2c00d21605f406d90c335256aa6cb1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:17.257545 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9b35bf9d2e9844b28803054c57783dd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.257575 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2c00d21605f406d90c335256aa6cb1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:17.257604 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b35bf9d2e9844b28803054c57783dd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.258496 28656 tablet_bootstrap.cc:492] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.258536 28656 ts_tablet_manager.cc:1367] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.258652 28656 raft_consensus.cc:348] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.258708 28656 raft_consensus.cc:374] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.258723 28656 raft_consensus.cc:729] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.258756 28656 consensus_queue.cc:260] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.258841 28656 ts_tablet_manager.cc:1398] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:17.259142 29740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:09:17.259326 29740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:09:17.259353 29740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240503 02:09:17.260632 27702 tablet_service.cc:1467] Processing CreateTablet for tablet e0b2d6d7518f44ba855a43086f24de4d (DEFAULT_TABLE table=default.Table_a2c00d21605f406d90c335256aa6cb1f [id=4a014d7616de49d99d5225b5a5c5c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.260733 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b2d6d7518f44ba855a43086f24de4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.260721 27427 tablet_service.cc:1467] Processing CreateTablet for tablet e0b2d6d7518f44ba855a43086f24de4d (DEFAULT_TABLE table=default.Table_a2c00d21605f406d90c335256aa6cb1f [id=4a014d7616de49d99d5225b5a5c5c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.260808 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b2d6d7518f44ba855a43086f24de4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.260874 27560 tablet_service.cc:1467] Processing CreateTablet for tablet e0b2d6d7518f44ba855a43086f24de4d (DEFAULT_TABLE table=default.Table_a2c00d21605f406d90c335256aa6cb1f [id=4a014d7616de49d99d5225b5a5c5c964]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.260948 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b2d6d7518f44ba855a43086f24de4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.261695 28655 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.261790 28664 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.262059 28656 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.262077 28655 tablet_bootstrap.cc:654] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.262116 28664 tablet_bootstrap.cc:654] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:17.262223 29740 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20240503 02:09:17.262255 29740 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20240503 02:09:17.262270 29740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:09:17.262295 29740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:09:17.262311 29740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:09:17.262326 29740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:09:17.262390 28656 tablet_bootstrap.cc:654] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.262615 29643 raft_consensus.cc:482] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.262667 29643 raft_consensus.cc:504] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.262763 29643 leader_election.cc:288] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.262840 28656 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.262882 28656 ts_tablet_manager.cc:1367] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.262959 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2081f9c133424955a57f3b9362c8fbd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.262960 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2081f9c133424955a57f3b9362c8fbd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.262985 28656 raft_consensus.cc:348] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.263021 27451 raft_consensus.cc:2454] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.263021 27583 raft_consensus.cc:2454] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.263032 28656 raft_consensus.cc:374] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.263070 28656 raft_consensus.cc:729] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.263095 28656 consensus_queue.cc:260] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.263132 27656 leader_election.cc:302] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.263173 28656 ts_tablet_manager.cc:1398] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.263226 29643 raft_consensus.cc:2781] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.263258 29643 raft_consensus.cc:482] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.263276 29643 raft_consensus.cc:3037] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.263833 29643 raft_consensus.cc:504] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.263952 29643 leader_election.cc:288] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.264096 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2081f9c133424955a57f3b9362c8fbd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.264094 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2081f9c133424955a57f3b9362c8fbd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.264158 27583 raft_consensus.cc:3037] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.264158 27451 raft_consensus.cc:3037] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.264174 29740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.117.20:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.26.117.62:37529,127.26.117.61:37847,127.26.117.60:36369
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.117.62:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.117.62
--webserver_port=40493
--never_fsync=true
--heap_profile_path=/tmp/kudu.29740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type RELEASE
built by None at 03 May 2024 01:43:14 UTC on efd1337a2aa3
build id 98
I20240503 02:09:17.264387 29740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:09:17.264622 29740 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:09:17.264747 27451 raft_consensus.cc:2454] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.264763 27583 raft_consensus.cc:2454] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.264953 27656 leader_election.cc:302] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.265030 29643 raft_consensus.cc:2781] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.265061 29643 raft_consensus.cc:686] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.265094 29643 consensus_queue.cc:237] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.265580 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 7282be27904d48f1af33a47693387a5d (DEFAULT_TABLE table=default.Table_9b35bf9d2e9844b28803054c57783dd6 [id=1d27a3c90ea145db9b5df89787fb782f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.265676 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7282be27904d48f1af33a47693387a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.265686 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 7282be27904d48f1af33a47693387a5d (DEFAULT_TABLE table=default.Table_9b35bf9d2e9844b28803054c57783dd6 [id=1d27a3c90ea145db9b5df89787fb782f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.265758 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7282be27904d48f1af33a47693387a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.266039 28774 consensus_queue.cc:1035] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.266734 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_59a0934fdeee428c9ff0236296e8d7a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.266839 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59a0934fdeee428c9ff0236296e8d7a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.267098 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7282be27904d48f1af33a47693387a5d (DEFAULT_TABLE table=default.Table_9b35bf9d2e9844b28803054c57783dd6 [id=1d27a3c90ea145db9b5df89787fb782f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.267197 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7282be27904d48f1af33a47693387a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.267637 28656 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.268076 28656 tablet_bootstrap.cc:654] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.268118 28655 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.268162 28655 ts_tablet_manager.cc:1367] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.268301 28655 raft_consensus.cc:348] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.268363 28655 raft_consensus.cc:374] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.268386 28655 raft_consensus.cc:729] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.268412 28655 consensus_queue.cc:260] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.268513 28656 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.268513 28655 ts_tablet_manager.cc:1398] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.268560 28656 ts_tablet_manager.cc:1367] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.268589 28655 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.268677 28656 raft_consensus.cc:348] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.268738 28656 raft_consensus.cc:374] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.268757 28656 raft_consensus.cc:729] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.268800 28656 consensus_queue.cc:260] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.268883 28656 ts_tablet_manager.cc:1398] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.268921 28655 tablet_bootstrap.cc:654] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.269116 27201 catalog_manager.cc:5484] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.269209 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.270886 28693 consensus_queue.cc:1035] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:17.271415 29816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:17.271715 29814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:09:17.272621 29813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:09:17.273845 27560 tablet_service.cc:1467] Processing CreateTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 (DEFAULT_TABLE table=default.Table_59a0934fdeee428c9ff0236296e8d7a1 [id=8f0f38ccfe5c453c8b2709eaa1036810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.273932 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf32b27ab1a428eb3b661db8578aaf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.273968 27427 tablet_service.cc:1467] Processing CreateTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 (DEFAULT_TABLE table=default.Table_59a0934fdeee428c9ff0236296e8d7a1 [id=8f0f38ccfe5c453c8b2709eaa1036810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.274053 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf32b27ab1a428eb3b661db8578aaf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.274365 28664 tablet_bootstrap.cc:492] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.274405 28664 ts_tablet_manager.cc:1367] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20240503 02:09:17.274541 28664 raft_consensus.cc:348] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.274591 28664 raft_consensus.cc:374] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.274608 28664 raft_consensus.cc:729] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.274653 28664 consensus_queue.cc:260] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.274741 28664 ts_tablet_manager.cc:1398] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.274821 28664 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.275144 28664 tablet_bootstrap.cc:654] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.275216 27702 tablet_service.cc:1467] Processing CreateTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 (DEFAULT_TABLE table=default.Table_59a0934fdeee428c9ff0236296e8d7a1 [id=8f0f38ccfe5c453c8b2709eaa1036810]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.275466 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf32b27ab1a428eb3b661db8578aaf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.275552 28656 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.275687 28664 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.275722 28664 ts_tablet_manager.cc:1367] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.275858 28656 tablet_bootstrap.cc:654] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.275842 28664 raft_consensus.cc:348] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.275900 28664 raft_consensus.cc:374] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.275918 28664 raft_consensus.cc:729] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.275976 28664 consensus_queue.cc:260] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.276069 28664 ts_tablet_manager.cc:1398] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.276273 27582 raft_consensus.cc:1261] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.276386 29740 server_base.cc:1034] running on GCE node
I20240503 02:09:17.276474 28740 consensus_queue.cc:1035] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.276523 28655 tablet_bootstrap.cc:492] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.276563 28655 ts_tablet_manager.cc:1367] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:17.276575 29740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:09:17.276691 28655 raft_consensus.cc:348] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.276738 28655 raft_consensus.cc:374] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.276757 28655 raft_consensus.cc:729] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.276790 28655 consensus_queue.cc:260] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.276818 29740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:09:17.276846 28656 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.276876 28656 ts_tablet_manager.cc:1367] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.276904 28655 ts_tablet_manager.cc:1398] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.276963 28655 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.276988 28656 raft_consensus.cc:348] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.277036 28656 raft_consensus.cc:374] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.277053 28656 raft_consensus.cc:729] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.277086 28656 consensus_queue.cc:260] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.277168 28656 ts_tablet_manager.cc:1398] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.277444 28655 tablet_bootstrap.cc:654] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.277559 27451 raft_consensus.cc:1261] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.277861 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3db8837099ba4da6aedfe7bb78c2b64a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.277966 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3db8837099ba4da6aedfe7bb78c2b64a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.278090 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_59a0934fdeee428c9ff0236296e8d7a1" } new_table_name: "default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5" modify_external_catalogs: true
I20240503 02:09:17.278216 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_730b70449f824837870ab1772a3e791b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.278283 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_730b70449f824837870ab1772a3e791b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.278424 28409 consensus_queue.cc:1035] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.278622 29558 tablet.cc:1722] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.278725 29643 consensus_queue.cc:1035] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.278914 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.278978 29566 tablet.cc:1722] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.279559 28710 tablet.cc:1722] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20240503 02:09:17.279748 27201 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bcf32b27ab1a428eb3b661db8578aaf6 consensus state has no leader
I20240503 02:09:17.282117 28664 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.282497 28664 tablet_bootstrap.cc:654] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.282800 27560 tablet_service.cc:1467] Processing CreateTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d (DEFAULT_TABLE table=default.Table_730b70449f824837870ab1772a3e791b [id=15574575df68415693e80eaea9955e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.282896 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1bbe9bd4a14f40ac340c5189f6056d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.283130 28664 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.283188 28664 ts_tablet_manager.cc:1367] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.283334 28664 raft_consensus.cc:348] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.283398 28664 raft_consensus.cc:374] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.283424 28664 raft_consensus.cc:729] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.283463 28664 consensus_queue.cc:260] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.283562 28664 ts_tablet_manager.cc:1398] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.283771 28655 tablet_bootstrap.cc:492] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.283814 28655 ts_tablet_manager.cc:1367] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:17.283888 28656 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.283965 28655 raft_consensus.cc:348] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.284022 28655 raft_consensus.cc:374] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.284040 28655 raft_consensus.cc:729] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.284070 28655 consensus_queue.cc:260] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.284160 28655 ts_tablet_manager.cc:1398] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.284215 28656 tablet_bootstrap.cc:654] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.284484 29740 hybrid_clock.cc:648] HybridClock initialized: now 1714702157283174 us; error 1326 us; skew 500 ppm
I20240503 02:09:17.284649 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d (DEFAULT_TABLE table=default.Table_730b70449f824837870ab1772a3e791b [id=15574575df68415693e80eaea9955e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.284673 28656 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.284706 28656 ts_tablet_manager.cc:1367] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.284727 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1bbe9bd4a14f40ac340c5189f6056d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.284818 28656 raft_consensus.cc:348] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.284865 28656 raft_consensus.cc:374] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.284880 28656 raft_consensus.cc:729] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.284929 28656 consensus_queue.cc:260] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.285004 28656 ts_tablet_manager.cc:1398] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.285589 29551 raft_consensus.cc:482] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.285636 29551 raft_consensus.cc:504] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.285657 27702 tablet_service.cc:1467] Processing CreateTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d (DEFAULT_TABLE table=default.Table_730b70449f824837870ab1772a3e791b [id=15574575df68415693e80eaea9955e92]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.285717 29551 leader_election.cc:288] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.285733 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1bbe9bd4a14f40ac340c5189f6056d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.286450 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b2d6d7518f44ba855a43086f24de4d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.286511 27583 raft_consensus.cc:2454] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.286777 28664 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.286787 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b2d6d7518f44ba855a43086f24de4d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.286875 27718 raft_consensus.cc:2454] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.287137 28664 tablet_bootstrap.cc:654] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.287228 27425 tablet_service.cc:1467] Processing CreateTablet for tablet be954e1ee239497fad20d030a835e2c1 (DEFAULT_TABLE table=default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.287302 27388 leader_election.cc:302] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.287369 28409 consensus_queue.cc:1035] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.287403 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be954e1ee239497fad20d030a835e2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.287439 28409 raft_consensus.cc:2781] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.287464 28409 raft_consensus.cc:482] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.287482 28409 raft_consensus.cc:3037] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.287667 27560 tablet_service.cc:1467] Processing CreateTablet for tablet be954e1ee239497fad20d030a835e2c1 (DEFAULT_TABLE table=default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.287752 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be954e1ee239497fad20d030a835e2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.288120 28409 raft_consensus.cc:504] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.288208 28409 leader_election.cc:288] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.288295 28655 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.288281 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e8d2612a1a8d4ff8b1913977101cf2ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.288364 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8d2612a1a8d4ff8b1913977101cf2ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.288653 28655 tablet_bootstrap.cc:654] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.288650 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b2d6d7518f44ba855a43086f24de4d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.288707 27722 raft_consensus.cc:3037] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.289352 27722 raft_consensus.cc:2454] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.289587 27388 leader_election.cc:302] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.289669 28685 raft_consensus.cc:2781] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.289911 28685 raft_consensus.cc:686] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.289949 28685 consensus_queue.cc:237] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.289995 27583 raft_consensus.cc:1261] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.290040 27451 raft_consensus.cc:1261] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.290068 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet bcf32b27ab1a428eb3b661db8578aaf6 (table default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5 [id=8f0f38ccfe5c453c8b2709eaa1036810]). Expected version 1 got 0
I20240503 02:09:17.290161 27202 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet bcf32b27ab1a428eb3b661db8578aaf6 (table default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5 [id=8f0f38ccfe5c453c8b2709eaa1036810]). Expected version 1 got 0
I20240503 02:09:17.290189 29240 raft_consensus.cc:482] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.290226 29240 raft_consensus.cc:504] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.290259 28656 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.290308 29240 leader_election.cc:288] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.290282 28740 consensus_queue.cc:1035] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.290328 29740 webserver.cc:460] Webserver started at http://127.26.117.62:40493/ using document root <none> and password file <none>
I20240503 02:09:17.290516 29740 fs_manager.cc:359] Metadata directory not provided
I20240503 02:09:17.290572 29740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:09:17.290583 28656 tablet_bootstrap.cc:654] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.290760 28740 consensus_queue.cc:1035] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.291239 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e3e21135574e9bba4f1c6896ee545c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.291319 27450 raft_consensus.cc:2454] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.291695 27202 catalog_manager.cc:5484] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.291841 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b2d6d7518f44ba855a43086f24de4d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.291899 27584 raft_consensus.cc:3037] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.292511 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e3e21135574e9bba4f1c6896ee545c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.292557 27584 raft_consensus.cc:2454] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.292577 27722 raft_consensus.cc:2454] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.292676 27523 leader_election.cc:302] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.292789 29240 raft_consensus.cc:2781] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.292822 29240 raft_consensus.cc:482] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.292840 29240 raft_consensus.cc:3037] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.293190 28656 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.293226 28656 ts_tablet_manager.cc:1367] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.293367 28656 raft_consensus.cc:348] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.293403 29240 raft_consensus.cc:504] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.293502 29240 leader_election.cc:288] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.293499 27702 tablet_service.cc:1467] Processing CreateTablet for tablet be954e1ee239497fad20d030a835e2c1 (DEFAULT_TABLE table=default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.293583 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be954e1ee239497fad20d030a835e2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.293414 28656 raft_consensus.cc:374] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.293661 28656 raft_consensus.cc:729] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.293697 28656 consensus_queue.cc:260] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.293780 28656 ts_tablet_manager.cc:1398] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.294024 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a0d6eed4bdb14fb7a777c4657a2279c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.294134 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0d6eed4bdb14fb7a777c4657a2279c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.294260 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e3e21135574e9bba4f1c6896ee545c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.294308 27722 raft_consensus.cc:3037] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.294853 27722 raft_consensus.cc:2454] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.295099 27523 leader_election.cc:302] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.295171 29240 raft_consensus.cc:2781] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.295208 29240 raft_consensus.cc:686] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.295240 29240 consensus_queue.cc:237] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.295436 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.296694 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 640547cbbff2492489dff09073733982 (DEFAULT_TABLE table=default.Table_e8d2612a1a8d4ff8b1913977101cf2ee [id=9db9ce96d7b644a190b654ea5442fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.296821 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.296816 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 640547cbbff2492489dff09073733982 (DEFAULT_TABLE table=default.Table_e8d2612a1a8d4ff8b1913977101cf2ee [id=9db9ce96d7b644a190b654ea5442fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.296914 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640547cbbff2492489dff09073733982. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.296898 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e3e21135574e9bba4f1c6896ee545c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.296967 27450 raft_consensus.cc:3037] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.297001 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 640547cbbff2492489dff09073733982 (DEFAULT_TABLE table=default.Table_e8d2612a1a8d4ff8b1913977101cf2ee [id=9db9ce96d7b644a190b654ea5442fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.297075 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640547cbbff2492489dff09073733982. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.297410 27450 raft_consensus.cc:2454] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.297959 28656 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.298213 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 640547cbbff2492489dff09073733982. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.298300 28656 tablet_bootstrap.cc:654] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.298812 28656 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.298854 28656 ts_tablet_manager.cc:1367] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.298979 28656 raft_consensus.cc:348] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.299031 28656 raft_consensus.cc:374] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.299072 28656 raft_consensus.cc:729] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.299104 28656 consensus_queue.cc:260] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.299185 28656 ts_tablet_manager.cc:1398] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.299815 29558 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.299947 29566 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.300207 27202 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.300202 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa (DEFAULT_TABLE table=default.Table_a0d6eed4bdb14fb7a777c4657a2279c0 [id=d619799c680f4c049324e1572598f632]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.300287 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.300253 27202 catalog_manager.cc:5484] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.301065 28710 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.301569 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa (DEFAULT_TABLE table=default.Table_a0d6eed4bdb14fb7a777c4657a2279c0 [id=d619799c680f4c049324e1572598f632]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.301579 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa (DEFAULT_TABLE table=default.Table_a0d6eed4bdb14fb7a777c4657a2279c0 [id=d619799c680f4c049324e1572598f632]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.301663 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.301664 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04ffc308a3b44eed8dce0a99d8f0a6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.302114 28656 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.302448 28656 tablet_bootstrap.cc:654] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.302979 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5" } modify_external_catalogs: true
I20240503 02:09:17.303040 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5" } modify_external_catalogs: true
I20240503 02:09:17.303380 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_81906986fe914a47b647cdaffb6e0880" } new_table_name: "default.Table_b44708e810b449d38c560721b1a7d665" modify_external_catalogs: true
I20240503 02:09:17.303761 28656 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.303798 28656 ts_tablet_manager.cc:1367] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:17.303941 28656 raft_consensus.cc:348] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.303998 28656 raft_consensus.cc:374] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.304018 28656 raft_consensus.cc:729] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.304055 28656 consensus_queue.cc:260] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.304152 28656 ts_tablet_manager.cc:1398] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.304836 29740 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:17.305006 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_9beb4cf015b3468a9aa10491b78c26da [id=53e0fca749ea4dc699450e4503b21c03]). Expected version 1 got 0
I20240503 02:09:17.305702 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet bcf32b27ab1a428eb3b661db8578aaf6
I20240503 02:09:17.305879 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.306063 28653 tablet_replica.cc:330] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.306113 28653 raft_consensus.cc:2229] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.306138 28653 raft_consensus.cc:2258] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.306193 29845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.306243 27584 raft_consensus.cc:1261] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.306285 28653 ts_tablet_manager.cc:1875] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.306315 29740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.306440 28655 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.306479 28655 ts_tablet_manager.cc:1367] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.001s
I20240503 02:09:17.306609 28655 raft_consensus.cc:348] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.306658 28655 raft_consensus.cc:374] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.306671 28655 raft_consensus.cc:729] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.306372 29740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data,/tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
uuid: "c324c9220fd14d3c817b18bbd3774439"
format_stamp: "Formatted at 2024-05-03 02:09:06 on dist-test-slave-r4f3"
I20240503 02:09:17.306689 28655 consensus_queue.cc:260] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.306710 29740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:09:17.306757 28655 ts_tablet_manager.cc:1398] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.306802 28655 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.307117 28655 tablet_bootstrap.cc:654] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.307456 28664 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.307487 28664 ts_tablet_manager.cc:1367] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.000s
I20240503 02:09:17.307564 28653 ts_tablet_manager.cc:1888] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.307606 28653 log.cc:1196] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/bcf32b27ab1a428eb3b661db8578aaf6
I20240503 02:09:17.307595 28664 raft_consensus.cc:348] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.307628 28664 raft_consensus.cc:374] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.307642 28664 raft_consensus.cc:729] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.307673 28664 consensus_queue.cc:260] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.307751 28653 ts_tablet_manager.cc:1909] T bcf32b27ab1a428eb3b661db8578aaf6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.307758 28664 ts_tablet_manager.cc:1398] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.308019 29240 raft_consensus.cc:482] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.308068 29240 raft_consensus.cc:504] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.308148 29240 leader_election.cc:288] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.308020 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.308291 28654 tablet_replica.cc:330] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.308328 28654 raft_consensus.cc:2229] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.308355 28654 raft_consensus.cc:2258] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.308490 28685 consensus_queue.cc:1035] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.308588 28654 ts_tablet_manager.cc:1875] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.308678 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf32b27ab1a428eb3b661db8578aaf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.309865 28654 ts_tablet_manager.cc:1888] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.309897 28654 log.cc:1196] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/bcf32b27ab1a428eb3b661db8578aaf6
I20240503 02:09:17.309993 28654 ts_tablet_manager.cc:1909] T bcf32b27ab1a428eb3b661db8578aaf6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.310289 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet bcf32b27ab1a428eb3b661db8578aaf6 (table default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5 [id=8f0f38ccfe5c453c8b2709eaa1036810]) successfully deleted
I20240503 02:09:17.310411 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25063c03d7dd4e7a87b7e8bd315684a8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.310473 27451 raft_consensus.cc:2454] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.310607 27524 leader_election.cc:302] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.310705 29240 raft_consensus.cc:2781] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.310740 29240 raft_consensus.cc:482] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.310757 29240 raft_consensus.cc:3037] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.310874 28664 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.311219 28664 tablet_bootstrap.cc:654] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.311313 29240 raft_consensus.cc:504] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.311440 29240 leader_election.cc:288] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.311477 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25063c03d7dd4e7a87b7e8bd315684a8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.311528 27722 raft_consensus.cc:2454] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.311569 28661 tablet_replica.cc:330] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.311618 28661 raft_consensus.cc:2229] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.311646 28661 raft_consensus.cc:2258] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.311959 28661 ts_tablet_manager.cc:1875] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.312346 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 837debf700f74dc680eb88168aeac413 (table default.Table_b44708e810b449d38c560721b1a7d665 [id=260be253b97d4a5681ac6d6875ad7414]). Expected version 1 got 0
I20240503 02:09:17.313251 28661 ts_tablet_manager.cc:1888] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.313302 28661 log.cc:1196] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/bcf32b27ab1a428eb3b661db8578aaf6
I20240503 02:09:17.313446 28661 ts_tablet_manager.cc:1909] T bcf32b27ab1a428eb3b661db8578aaf6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.313544 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25063c03d7dd4e7a87b7e8bd315684a8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.313586 27722 raft_consensus.cc:3037] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.313814 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet bcf32b27ab1a428eb3b661db8578aaf6 (table default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5 [id=8f0f38ccfe5c453c8b2709eaa1036810]) successfully deleted
I20240503 02:09:17.314067 27722 raft_consensus.cc:2454] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.314193 27523 leader_election.cc:302] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.314215 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25063c03d7dd4e7a87b7e8bd315684a8" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.314266 29240 raft_consensus.cc:2781] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.314267 27451 raft_consensus.cc:3037] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.314296 29240 raft_consensus.cc:686] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.314332 29240 consensus_queue.cc:237] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.314379 28710 tablet.cc:1722] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.314851 27451 raft_consensus.cc:2454] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.315367 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ec955a8689124c99a40190db45718286"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.315474 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec955a8689124c99a40190db45718286 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.315498 27195 catalog_manager.cc:5484] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.315639 27722 raft_consensus.cc:1261] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.315714 29566 tablet.cc:1722] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.315905 29046 mvcc.cc:205] Tried to move back new op lower bound from 7023420036325101568 to 7023420035429732352. Current Snapshot: MvccSnapshot[applied={T|T < 7023420036325101568}]
I20240503 02:09:17.316387 29550 consensus_queue.cc:1035] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.316474 29550 consensus_queue.cc:1035] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.316812 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_28e4b24a6e884bb6a60d26a9ae69d82a" } new_table_name: "default.Table_299f9beef20b43108d34fe3b4ef1a355" modify_external_catalogs: true
I20240503 02:09:17.317198 29558 tablet.cc:1722] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.318333 28655 tablet_bootstrap.cc:492] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.318465 28655 ts_tablet_manager.cc:1367] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240503 02:09:17.318840 28655 raft_consensus.cc:348] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.319007 28655 raft_consensus.cc:374] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.319064 28655 raft_consensus.cc:729] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.319168 28655 consensus_queue.cc:260] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.319371 28655 ts_tablet_manager.cc:1398] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.319520 28655 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.320827 28655 tablet_bootstrap.cc:654] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.321113 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_730b70449f824837870ab1772a3e791b" } new_table_name: "default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79" modify_external_catalogs: true
I20240503 02:09:17.321568 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet bcf32b27ab1a428eb3b661db8578aaf6 (table default.Table_99b9a728c57f47a6b89c42f8dc7a7bb5 [id=8f0f38ccfe5c453c8b2709eaa1036810]) successfully deleted
I20240503 02:09:17.322280 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_b44708e810b449d38c560721b1a7d665 [id=260be253b97d4a5681ac6d6875ad7414] alter complete (version 1)
I20240503 02:09:17.322292 28655 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.322328 28655 ts_tablet_manager.cc:1367] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240503 02:09:17.322460 28655 raft_consensus.cc:348] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.322515 28655 raft_consensus.cc:374] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.322532 28655 raft_consensus.cc:729] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.322561 28655 consensus_queue.cc:260] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.322644 28655 ts_tablet_manager.cc:1398] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.322695 28655 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.323014 28655 tablet_bootstrap.cc:654] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:17.323136 27198 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad1bbe9bd4a14f40ac340c5189f6056d consensus state has no leader
I20240503 02:09:17.323462 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 50566b0702db4f85b581931f415ca42d (DEFAULT_TABLE table=default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.323544 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50566b0702db4f85b581931f415ca42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.325153 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 50566b0702db4f85b581931f415ca42d (DEFAULT_TABLE table=default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.325241 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50566b0702db4f85b581931f415ca42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.325932 28710 tablet.cc:1722] T b3eb52c8c0a94673bf1546f71320a6fe P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.325961 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79 [id=15574575df68415693e80eaea9955e92]). Expected version 1 got 0
I20240503 02:09:17.326139 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 50566b0702db4f85b581931f415ca42d (DEFAULT_TABLE table=default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.326433 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50566b0702db4f85b581931f415ca42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.330116 28656 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.330308 29558 tablet.cc:1722] T b3eb52c8c0a94673bf1546f71320a6fe P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.330544 28656 tablet_bootstrap.cc:654] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.331053 29566 tablet.cc:1722] T b3eb52c8c0a94673bf1546f71320a6fe P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.331622 29740 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:09:17.331872 29740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:09:17.331976 29740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:09:17.333426 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 1fa99492baf64897afd77f6466ae805c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.334120 28664 tablet_bootstrap.cc:492] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.334164 28664 ts_tablet_manager.cc:1367] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.000s
I20240503 02:09:17.334331 28664 raft_consensus.cc:348] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.334389 28664 raft_consensus.cc:374] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.334407 28664 raft_consensus.cc:729] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.334440 28664 consensus_queue.cc:260] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.334524 28664 ts_tablet_manager.cc:1398] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.334590 28409 raft_consensus.cc:482] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.334627 28409 raft_consensus.cc:504] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.334568 28664 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.334712 28409 leader_election.cc:288] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.335019 28664 tablet_bootstrap.cc:654] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.334996 27202 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_299f9beef20b43108d34fe3b4ef1a355 [id=7515b8bcce32455fac410e29a6b35adf] alter complete (version 1)
I20240503 02:09:17.335109 28409 raft_consensus.cc:482] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.335146 28409 raft_consensus.cc:504] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.335237 28409 leader_election.cc:288] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.335430 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.335500 27584 raft_consensus.cc:2454] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.335561 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.335593 27386 leader_election.cc:302] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.335652 28409 raft_consensus.cc:2781] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.335675 28409 raft_consensus.cc:482] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.335690 28409 raft_consensus.cc:3037] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.336026 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1fa99492baf64897afd77f6466ae805c
I20240503 02:09:17.336114 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 1fa99492baf64897afd77f6466ae805c deleted and replaced by tablet eb76d46144e74642bdcb0146ed419ce5
I20240503 02:09:17.336277 28409 raft_consensus.cc:504] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.336376 28409 leader_election.cc:288] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
W20240503 02:09:17.336519 27388 leader_election.cc:341] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.336673 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1bbe9bd4a14f40ac340c5189f6056d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.336691 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.336737 27583 raft_consensus.cc:2454] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.336757 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.336796 27584 raft_consensus.cc:3037] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.336807 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1bbe9bd4a14f40ac340c5189f6056d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.336854 27720 raft_consensus.cc:2454] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:17.336988 27388 leader_election.cc:341] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.337062 27388 leader_election.cc:302] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.337160 29550 raft_consensus.cc:2781] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.337191 29550 raft_consensus.cc:482] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.337200 28655 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.337239 28655 ts_tablet_manager.cc:1367] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20240503 02:09:17.337208 29550 raft_consensus.cc:3037] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.337421 27584 raft_consensus.cc:2454] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.337746 28655 raft_consensus.cc:348] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.337816 28655 raft_consensus.cc:374] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.337836 28655 raft_consensus.cc:729] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.337869 28655 consensus_queue.cc:260] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.337973 28655 ts_tablet_manager.cc:1398] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.337982 29550 raft_consensus.cc:504] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.338110 27202 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a81c565ca83042d6bd00fc3c7cb4eb3c" } new_table_name: "default.Table_d6c7af51340244c2b242eab1f705a0d8" modify_external_catalogs: true
I20240503 02:09:17.338152 29550 leader_election.cc:288] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.338215 28655 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.338549 28655 tablet_bootstrap.cc:654] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.338532 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_59f76355e5b649b68043222a5d0cd38a" } modify_external_catalogs: true
I20240503 02:09:17.338585 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_59f76355e5b649b68043222a5d0cd38a" } modify_external_catalogs: true
I20240503 02:09:17.339012 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa99492baf64897afd77f6466ae805c with delete_type TABLET_DATA_DELETED (replaced by tablet eb76d46144e74642bdcb0146ed419ce5) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.339097 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa99492baf64897afd77f6466ae805c with delete_type TABLET_DATA_DELETED (replaced by tablet eb76d46144e74642bdcb0146ed419ce5) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.339115 28661 tablet_replica.cc:330] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.339174 28654 tablet_replica.cc:330] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.339186 28661 raft_consensus.cc:2229] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.339227 28654 raft_consensus.cc:2229] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.339237 28661 raft_consensus.cc:2258] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.339257 28654 raft_consensus.cc:2258] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.339341 28661 ts_tablet_manager.cc:1875] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.339354 28654 ts_tablet_manager.cc:1875] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.339748 29740 rpc_server.cc:307] RPC server started. Bound to: 127.26.117.62:37529
I20240503 02:09:17.340119 29740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/master-0/data/info.pb
I20240503 02:09:17.340585 28661 ts_tablet_manager.cc:1888] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.340628 28661 log.cc:1196] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1fa99492baf64897afd77f6466ae805c
I20240503 02:09:17.340690 28654 ts_tablet_manager.cc:1888] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.340726 28654 log.cc:1196] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1fa99492baf64897afd77f6466ae805c
I20240503 02:09:17.340869 28661 ts_tablet_manager.cc:1909] T 1fa99492baf64897afd77f6466ae805c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.340886 28654 ts_tablet_manager.cc:1909] T 1fa99492baf64897afd77f6466ae805c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.341171 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]) successfully deleted
I20240503 02:09:17.341266 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 9181c05524c3474282c7f526e1195d29 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.341313 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]) successfully deleted
I20240503 02:09:17.341342 28656 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.341382 28656 ts_tablet_manager.cc:1367] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:17.341495 28656 raft_consensus.cc:348] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.341547 28656 raft_consensus.cc:374] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.341565 28656 raft_consensus.cc:729] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.341601 28656 consensus_queue.cc:260] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.341650 28655 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.341688 28655 ts_tablet_manager.cc:1367] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:17.341696 28656 ts_tablet_manager.cc:1398] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.341804 28655 raft_consensus.cc:348] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.341854 28655 raft_consensus.cc:374] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.341873 28655 raft_consensus.cc:729] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.341897 28655 consensus_queue.cc:260] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.341907 27386 leader_election.cc:302] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters: c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.341979 28655 ts_tablet_manager.cc:1398] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.342082 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1bbe9bd4a14f40ac340c5189f6056d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.342135 27582 raft_consensus.cc:3037] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.342352 29908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.117.62:37529 every 8 connection(s)
I20240503 02:09:17.342517 28766 raft_consensus.cc:2781] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.342559 28766 raft_consensus.cc:686] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.342594 28766 consensus_queue.cc:237] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.342757 27582 raft_consensus.cc:2454] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.342962 27386 leader_election.cc:302] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.342970 29909 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:09:17.342888 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1bbe9bd4a14f40ac340c5189f6056d" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.343051 27722 raft_consensus.cc:3037] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.343592 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa99492baf64897afd77f6466ae805c with delete_type TABLET_DATA_DELETED (replaced by tablet eb76d46144e74642bdcb0146ed419ce5) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.343639 27722 raft_consensus.cc:2454] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.343820 28653 tablet_replica.cc:330] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.343888 28653 raft_consensus.cc:2229] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.343921 28653 raft_consensus.cc:2258] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.344020 28653 ts_tablet_manager.cc:1875] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.344192 28767 raft_consensus.cc:2781] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.344233 28767 raft_consensus.cc:686] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.344264 28767 consensus_queue.cc:237] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.345633 27092 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskpsd1k2/build/release/bin/kudu as pid 29740
I20240503 02:09:17.346000 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55" } modify_external_catalogs: true
I20240503 02:09:17.346055 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55" } modify_external_catalogs: true
I20240503 02:09:17.346518 28664 tablet_bootstrap.cc:492] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.346556 28664 ts_tablet_manager.cc:1367] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:17.343994 28409 consensus_queue.cc:1035] T e91b8d143d5c49c19d4a328e714b572d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.346740 28664 raft_consensus.cc:348] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.346793 28664 raft_consensus.cc:374] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.346810 28664 raft_consensus.cc:729] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.346844 28653 ts_tablet_manager.cc:1888] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.346849 28664 consensus_queue.cc:260] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.346889 28653 log.cc:1196] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1fa99492baf64897afd77f6466ae805c
I20240503 02:09:17.346938 28664 ts_tablet_manager.cc:1398] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.346997 28664 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.347071 28653 ts_tablet_manager.cc:1909] T 1fa99492baf64897afd77f6466ae805c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.347396 28664 tablet_bootstrap.cc:654] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.347576 27560 tablet_service.cc:1467] Processing CreateTablet for tablet eb76d46144e74642bdcb0146ed419ce5 (DEFAULT_TABLE table=default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.347630 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8a8d8edf43434e5597e58084babb7843
I20240503 02:09:17.347668 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb76d46144e74642bdcb0146ed419ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.347715 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9181c05524c3474282c7f526e1195d29
I20240503 02:09:17.347755 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 9181c05524c3474282c7f526e1195d29 deleted and replaced by tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab
I20240503 02:09:17.347811 27198 catalog_manager.cc:5484] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.347885 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79 [id=15574575df68415693e80eaea9955e92]). Expected version 1 got 0
I20240503 02:09:17.347914 27198 catalog_manager.cc:5484] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.348326 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8d8edf43434e5597e58084babb7843 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.348368 27702 tablet_service.cc:1467] Processing CreateTablet for tablet eb76d46144e74642bdcb0146ed419ce5 (DEFAULT_TABLE table=default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.348395 28661 tablet_replica.cc:330] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.348592 28661 raft_consensus.cc:2229] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.348624 28661 raft_consensus.cc:2258] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.348672 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8d8edf43434e5597e58084babb7843 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.348745 28656 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.348888 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 9181c05524c3474282c7f526e1195d29 with delete_type TABLET_DATA_DELETED (replaced by tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.349097 28656 tablet_bootstrap.cc:654] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.349187 28685 raft_consensus.cc:482] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.349256 28685 raft_consensus.cc:504] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.349411 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 9181c05524c3474282c7f526e1195d29 with delete_type TABLET_DATA_DELETED (replaced by tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.349411 28685 leader_election.cc:288] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.349686 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1fa99492baf64897afd77f6466ae805c (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]) successfully deleted
I20240503 02:09:17.349730 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d0dfb8e744d0ca7044f0c03835a90" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.349807 27583 raft_consensus.cc:2454] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.349900 27427 tablet_service.cc:1467] Processing CreateTablet for tablet eb76d46144e74642bdcb0146ed419ce5 (DEFAULT_TABLE table=default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.349956 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 9181c05524c3474282c7f526e1195d29 with delete_type TABLET_DATA_DELETED (replaced by tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.349985 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb76d46144e74642bdcb0146ed419ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.350036 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb76d46144e74642bdcb0146ed419ce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.350471 28654 tablet_replica.cc:330] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.350490 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8d8edf43434e5597e58084babb7843 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.350535 28654 raft_consensus.cc:2229] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.350569 28654 raft_consensus.cc:2258] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.350957 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d0dfb8e744d0ca7044f0c03835a90" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.351019 28661 ts_tablet_manager.cc:1875] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.351022 27720 raft_consensus.cc:2454] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.352293 28661 ts_tablet_manager.cc:1888] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.352334 28661 log.cc:1196] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8a8d8edf43434e5597e58084babb7843
I20240503 02:09:17.352505 28661 ts_tablet_manager.cc:1909] T 8a8d8edf43434e5597e58084babb7843 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.352808 28661 tablet_replica.cc:330] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.352813 27388 leader_election.cc:302] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.352864 28661 raft_consensus.cc:2229] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.352905 28661 raft_consensus.cc:2258] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.352921 29550 raft_consensus.cc:2781] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.352949 29550 raft_consensus.cc:482] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.352967 29550 raft_consensus.cc:3037] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.353127 28661 ts_tablet_manager.cc:1875] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.353569 29550 raft_consensus.cc:504] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.353667 29550 leader_election.cc:288] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.353785 28654 ts_tablet_manager.cc:1875] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.353893 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8a8d8edf43434e5597e58084babb7843 (table default.Table_59f76355e5b649b68043222a5d0cd38a [id=da76a80d32d34ee0b39f826d114a51ac]) successfully deleted
I20240503 02:09:17.354094 29558 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.354346 28661 ts_tablet_manager.cc:1888] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.354386 28661 log.cc:1196] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/9181c05524c3474282c7f526e1195d29
I20240503 02:09:17.354559 28661 ts_tablet_manager.cc:1909] T 9181c05524c3474282c7f526e1195d29 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.354859 28656 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.354897 28656 ts_tablet_manager.cc:1367] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.355029 28654 ts_tablet_manager.cc:1888] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.355015 28656 raft_consensus.cc:348] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.355052 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet bb1fd17aa4a24b3a903c71482333ec84
I20240503 02:09:17.355062 28654 log.cc:1196] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8a8d8edf43434e5597e58084babb7843
I20240503 02:09:17.355250 28654 ts_tablet_manager.cc:1909] T 8a8d8edf43434e5597e58084babb7843 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.355378 28655 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.355510 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.355584 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 9181c05524c3474282c7f526e1195d29 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:17.355676 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.355706 28655 tablet_bootstrap.cc:654] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.355746 28661 tablet_replica.cc:330] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.355809 28661 raft_consensus.cc:2229] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.355842 28661 raft_consensus.cc:2258] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.355950 28710 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.356092 28661 ts_tablet_manager.cc:1875] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.356263 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet bb1fd17aa4a24b3a903c71482333ec84 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.356766 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d0dfb8e744d0ca7044f0c03835a90" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.356817 27584 raft_consensus.cc:3037] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.357414 27584 raft_consensus.cc:2454] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.357425 28661 ts_tablet_manager.cc:1888] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.357460 28661 log.cc:1196] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/bb1fd17aa4a24b3a903c71482333ec84
I20240503 02:09:17.357488 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "051d0dfb8e744d0ca7044f0c03835a90" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.357534 27722 raft_consensus.cc:3037] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.357630 28661 ts_tablet_manager.cc:1909] T bb1fd17aa4a24b3a903c71482333ec84 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.355062 28656 raft_consensus.cc:374] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.357671 28656 raft_consensus.cc:729] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.357713 28656 consensus_queue.cc:260] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.357815 28656 ts_tablet_manager.cc:1398] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.358093 27722 raft_consensus.cc:2454] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.358242 27388 leader_election.cc:302] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.358566 28685 raft_consensus.cc:2781] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.358609 28685 raft_consensus.cc:686] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.358645 28685 consensus_queue.cc:237] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.358945 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]) successfully deleted
I20240503 02:09:17.359592 29550 consensus_queue.cc:1035] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.360494 27196 master_service.cc:946] ReplaceTablet: received request to replace tablet 33d313988a8c418ea82af994d23e7e15 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.361053 28685 consensus_queue.cc:1035] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.361979 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.362207 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.362296 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.363065 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.363152 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.364339 28655 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.364380 28655 ts_tablet_manager.cc:1367] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:17.364504 28655 raft_consensus.cc:348] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.364557 28655 raft_consensus.cc:374] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.364574 28655 raft_consensus.cc:729] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.364599 28655 consensus_queue.cc:260] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.364681 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 33d313988a8c418ea82af994d23e7e15
I20240503 02:09:17.364688 28655 ts_tablet_manager.cc:1398] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.364706 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d6c7af51340244c2b242eab1f705a0d8 [id=5ff15871699b4869b63a0187224a6cd7] alter complete (version 3)
I20240503 02:09:17.364737 28655 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.364754 27196 catalog_manager.cc:6554] ReplaceTablet: tablet 33d313988a8c418ea82af994d23e7e15 deleted and replaced by tablet f42c6ab1499d4b058241376b6f211bde
I20240503 02:09:17.365069 28655 tablet_bootstrap.cc:654] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.365229 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d313988a8c418ea82af994d23e7e15 with delete_type TABLET_DATA_DELETED (replaced by tablet f42c6ab1499d4b058241376b6f211bde) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.365921 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d313988a8c418ea82af994d23e7e15 with delete_type TABLET_DATA_DELETED (replaced by tablet f42c6ab1499d4b058241376b6f211bde) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.366271 28664 tablet_bootstrap.cc:492] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.366333 28664 ts_tablet_manager.cc:1367] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.000s
I20240503 02:09:17.366454 28664 raft_consensus.cc:348] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.366500 28664 raft_consensus.cc:374] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.366515 28664 raft_consensus.cc:729] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.366555 28664 consensus_queue.cc:260] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.366650 28664 ts_tablet_manager.cc:1398] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.366703 28664 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.366747 29566 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.367008 28664 tablet_bootstrap.cc:654] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.367348 27560 tablet_service.cc:1467] Processing CreateTablet for tablet f42c6ab1499d4b058241376b6f211bde (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.367960 27197 catalog_manager.cc:5484] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.368064 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.368083 27703 tablet_service.cc:1467] Processing CreateTablet for tablet f42c6ab1499d4b058241376b6f211bde (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.368162 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f42c6ab1499d4b058241376b6f211bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.368425 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 949b2d82e23340d3b4ce56934a4db493 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.368515 28655 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.368552 28655 ts_tablet_manager.cc:1367] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.368666 28655 raft_consensus.cc:348] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.368708 27427 tablet_service.cc:1467] Processing CreateTablet for tablet f42c6ab1499d4b058241376b6f211bde (DEFAULT_TABLE table=default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.368800 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f42c6ab1499d4b058241376b6f211bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.368716 28655 raft_consensus.cc:374] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.368850 28655 raft_consensus.cc:729] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.368887 28655 consensus_queue.cc:260] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.368973 28655 ts_tablet_manager.cc:1398] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.369938 28655 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:17.370112 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet eb76d46144e74642bdcb0146ed419ce5 consensus state has no leader
I20240503 02:09:17.370304 28655 tablet_bootstrap.cc:654] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.370368 27196 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.372867 28654 tablet_replica.cc:330] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.372936 28654 raft_consensus.cc:2229] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.372967 28654 raft_consensus.cc:2258] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.373001 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.373082 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8a8d8edf43434e5597e58084babb7843 (table default.Table_59f76355e5b649b68043222a5d0cd38a [id=da76a80d32d34ee0b39f826d114a51ac]) successfully deleted
I20240503 02:09:17.373217 28654 ts_tablet_manager.cc:1875] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.373970 28653 tablet_replica.cc:330] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.374049 28653 raft_consensus.cc:2229] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.374114 28653 raft_consensus.cc:2258] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.374286 28653 ts_tablet_manager.cc:1875] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.374279 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 33d313988a8c418ea82af994d23e7e15 with delete_type TABLET_DATA_DELETED (replaced by tablet f42c6ab1499d4b058241376b6f211bde) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.374500 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f42c6ab1499d4b058241376b6f211bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.375026 27770 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.375849 28656 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.375854 28653 ts_tablet_manager.cc:1888] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.375910 28653 log.cc:1196] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9181c05524c3474282c7f526e1195d29
I20240503 02:09:17.375967 28661 tablet_replica.cc:330] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.376082 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 949b2d82e23340d3b4ce56934a4db493
I20240503 02:09:17.376158 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 949b2d82e23340d3b4ce56934a4db493 deleted and replaced by tablet 53b8f4c16c99410286f69fef3236e920
I20240503 02:09:17.376135 28661 raft_consensus.cc:2229] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.376190 28661 raft_consensus.cc:2258] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.376226 28656 tablet_bootstrap.cc:654] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.376282 29240 raft_consensus.cc:482] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.376324 29240 raft_consensus.cc:504] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.376349 28661 ts_tablet_manager.cc:1875] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.376448 29240 leader_election.cc:288] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:17.376500 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.376749 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be954e1ee239497fad20d030a835e2c1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.376812 27451 raft_consensus.cc:2454] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.377012 27524 leader_election.cc:302] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.377082 29240 raft_consensus.cc:2781] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.377105 29240 raft_consensus.cc:482] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.377122 29240 raft_consensus.cc:3037] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.377106 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 203665395f3745f196fb393dd82b8fc9 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.377772 29240 raft_consensus.cc:504] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.377856 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 949b2d82e23340d3b4ce56934a4db493 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b8f4c16c99410286f69fef3236e920) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.377884 29240 leader_election.cc:288] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:17.377930 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 949b2d82e23340d3b4ce56934a4db493 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b8f4c16c99410286f69fef3236e920) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.378250 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be954e1ee239497fad20d030a835e2c1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.378307 27451 raft_consensus.cc:3037] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.378587 28661 ts_tablet_manager.cc:1888] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.378628 28661 log.cc:1196] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/33d313988a8c418ea82af994d23e7e15
I20240503 02:09:17.378808 28661 ts_tablet_manager.cc:1909] T 33d313988a8c418ea82af994d23e7e15 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.378827 28656 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.378867 28656 ts_tablet_manager.cc:1367] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.378917 27451 raft_consensus.cc:2454] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.378935 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be954e1ee239497fad20d030a835e2c1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.378985 28656 raft_consensus.cc:348] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.379022 27720 raft_consensus.cc:3037] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.379040 28656 raft_consensus.cc:374] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.379057 28656 raft_consensus.cc:729] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.379087 28656 consensus_queue.cc:260] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.379199 28656 ts_tablet_manager.cc:1398] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.379266 27524 leader_election.cc:302] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.379359 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be954e1ee239497fad20d030a835e2c1" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.379649 27720 raft_consensus.cc:2454] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.379745 28507 raft_consensus.cc:2781] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.379787 28507 raft_consensus.cc:686] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.379824 28507 consensus_queue.cc:237] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.379938 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 53b8f4c16c99410286f69fef3236e920 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.380311 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 53b8f4c16c99410286f69fef3236e920 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.380400 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b8f4c16c99410286f69fef3236e920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.380558 28653 ts_tablet_manager.cc:1909] T 9181c05524c3474282c7f526e1195d29 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.380630 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 33d313988a8c418ea82af994d23e7e15 (table default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:17.380757 28653 tablet_replica.cc:330] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.380887 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 53b8f4c16c99410286f69fef3236e920 (DEFAULT_TABLE table=default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.380995 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b8f4c16c99410286f69fef3236e920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.381076 28653 raft_consensus.cc:2229] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.381143 28653 raft_consensus.cc:2258] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.381338 28653 ts_tablet_manager.cc:1875] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.381462 28654 ts_tablet_manager.cc:1888] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.381502 28654 log.cc:1196] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9181c05524c3474282c7f526e1195d29
I20240503 02:09:17.381702 27197 catalog_manager.cc:5484] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.381672 28654 ts_tablet_manager.cc:1909] T 9181c05524c3474282c7f526e1195d29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.382354 28507 raft_consensus.cc:482] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.382378 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9181c05524c3474282c7f526e1195d29 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:17.382400 28507 raft_consensus.cc:504] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.382490 28507 leader_election.cc:288] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.382570 28656 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.382602 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
W20240503 02:09:17.382786 27523 leader_election.cc:341] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.382890 28656 tablet_bootstrap.cc:654] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.383061 28653 ts_tablet_manager.cc:1888] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.383102 28653 log.cc:1196] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8a8d8edf43434e5597e58084babb7843
I20240503 02:09:17.383131 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53b8f4c16c99410286f69fef3236e920. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.383271 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9181c05524c3474282c7f526e1195d29 (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:17.383316 28653 ts_tablet_manager.cc:1909] T 8a8d8edf43434e5597e58084babb7843 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.383353 28654 tablet_replica.cc:330] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.383404 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.383400 28654 raft_consensus.cc:2229] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.383456 27451 raft_consensus.cc:2454] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.383461 28654 raft_consensus.cc:2258] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.383509 28653 tablet_replica.cc:330] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.383556 28653 raft_consensus.cc:2229] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.383564 28654 ts_tablet_manager.cc:1875] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.383584 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8a8d8edf43434e5597e58084babb7843 (table default.Table_59f76355e5b649b68043222a5d0cd38a [id=da76a80d32d34ee0b39f826d114a51ac]) successfully deleted
I20240503 02:09:17.383605 28653 raft_consensus.cc:2258] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.383652 27524 leader_election.cc:302] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters: c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.383700 28653 ts_tablet_manager.cc:1875] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.383728 29240 raft_consensus.cc:2781] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.383752 29240 raft_consensus.cc:482] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.383769 29240 raft_consensus.cc:3037] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.384292 29240 raft_consensus.cc:504] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.384408 29240 leader_election.cc:288] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.384617 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.384727 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.384776 27451 raft_consensus.cc:3037] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:17.384763 27523 leader_election.cc:341] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.385053 28653 ts_tablet_manager.cc:1888] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.385092 28653 log.cc:1196] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/bb1fd17aa4a24b3a903c71482333ec84
I20240503 02:09:17.385217 28654 ts_tablet_manager.cc:1888] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.385260 28654 log.cc:1196] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/bb1fd17aa4a24b3a903c71482333ec84
I20240503 02:09:17.385265 28653 ts_tablet_manager.cc:1909] T bb1fd17aa4a24b3a903c71482333ec84 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.385308 27451 raft_consensus.cc:2454] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.385471 27524 leader_election.cc:302] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters: c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.385488 28654 ts_tablet_manager.cc:1909] T bb1fd17aa4a24b3a903c71482333ec84 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.385563 29240 raft_consensus.cc:2781] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.385596 29240 raft_consensus.cc:686] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.385629 29240 consensus_queue.cc:237] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.385740 28654 tablet_replica.cc:330] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.385807 28654 raft_consensus.cc:2229] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.385835 28654 raft_consensus.cc:2258] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.385929 28654 ts_tablet_manager.cc:1875] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.386024 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]) successfully deleted
I20240503 02:09:17.386096 28653 tablet_replica.cc:330] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.386298 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet bb1fd17aa4a24b3a903c71482333ec84 (table default.Table_f59c64e0ba934ea9a5e5b40bf72bdb55 [id=846642e389d8455184d0ce48466fcbf4]) successfully deleted
I20240503 02:09:17.386581 28653 raft_consensus.cc:2229] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.386651 28653 raft_consensus.cc:2258] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.386746 28653 ts_tablet_manager.cc:1875] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.386729 27197 catalog_manager.cc:5484] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.386816 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 949b2d82e23340d3b4ce56934a4db493 with delete_type TABLET_DATA_DELETED (replaced by tablet 53b8f4c16c99410286f69fef3236e920) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.386911 28661 tablet_replica.cc:330] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.386965 28661 raft_consensus.cc:2229] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.386991 28661 raft_consensus.cc:2258] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.387454 28654 ts_tablet_manager.cc:1888] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.387501 28654 log.cc:1196] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/33d313988a8c418ea82af994d23e7e15
I20240503 02:09:17.387683 28654 ts_tablet_manager.cc:1909] T 33d313988a8c418ea82af994d23e7e15 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.387914 28656 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.387928 28654 tablet_replica.cc:330] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.387954 28656 ts_tablet_manager.cc:1367] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:17.387996 28654 raft_consensus.cc:2229] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.388018 28654 pending_rounds.cc:62] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:17.388032 28654 pending_rounds.cc:68] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420034524921856 op_type: NO_OP noop_request { }
I20240503 02:09:17.388062 28654 raft_consensus.cc:2866] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.388074 28656 raft_consensus.cc:348] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.388132 28656 raft_consensus.cc:374] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.388149 28656 raft_consensus.cc:729] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.388175 28656 consensus_queue.cc:260] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.388257 28656 ts_tablet_manager.cc:1398] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.388403 28653 ts_tablet_manager.cc:1888] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.388517 28653 log.cc:1196] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/33d313988a8c418ea82af994d23e7e15
I20240503 02:09:17.388594 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 33d313988a8c418ea82af994d23e7e15 (table default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:17.388705 28653 ts_tablet_manager.cc:1909] T 33d313988a8c418ea82af994d23e7e15 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.388082 28654 raft_consensus.cc:2258] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.389007 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 33d313988a8c418ea82af994d23e7e15 (table default.Table_b461ce136ff84b40a82e638866255964 [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:17.389079 28653 tablet_replica.cc:330] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.389118 28653 raft_consensus.cc:2229] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.389356 28653 raft_consensus.cc:2258] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.389571 28653 ts_tablet_manager.cc:1875] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.389734 27500 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.390817 28653 ts_tablet_manager.cc:1888] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.390862 28653 log.cc:1196] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/949b2d82e23340d3b4ce56934a4db493
I20240503 02:09:17.391054 28653 ts_tablet_manager.cc:1909] T 949b2d82e23340d3b4ce56934a4db493 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.391358 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 949b2d82e23340d3b4ce56934a4db493 (table default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]) successfully deleted
I20240503 02:09:17.392336 27582 raft_consensus.cc:1261] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.392436 28656 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.392670 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.392771 28656 tablet_bootstrap.cc:654] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.393944 28661 ts_tablet_manager.cc:1875] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.394115 29550 consensus_queue.cc:1035] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.395241 28710 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.395275 28661 ts_tablet_manager.cc:1888] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.395323 28661 log.cc:1196] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/949b2d82e23340d3b4ce56934a4db493
I20240503 02:09:17.395514 28661 ts_tablet_manager.cc:1909] T 949b2d82e23340d3b4ce56934a4db493 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.396097 29566 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.396394 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 949b2d82e23340d3b4ce56934a4db493 (table default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]) successfully deleted
I20240503 02:09:17.398929 27500 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.399835 28740 consensus_queue.cc:1035] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.400317 27720 raft_consensus.cc:1261] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.400478 28767 consensus_queue.cc:1035] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.400612 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap starting.
I20240503 02:09:17.400705 28656 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.400750 28656 ts_tablet_manager.cc:1367] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:17.400880 28656 raft_consensus.cc:348] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.400954 28656 raft_consensus.cc:374] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.400974 28656 raft_consensus.cc:729] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.401008 28656 consensus_queue.cc:260] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.401113 28656 ts_tablet_manager.cc:1398] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.401419 29558 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.402040 27196 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79 [id=15574575df68415693e80eaea9955e92]). Expected version 1 got 0
I20240503 02:09:17.402189 28767 raft_consensus.cc:482] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.402232 28767 raft_consensus.cc:504] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.402334 28767 leader_election.cc:288] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.403017 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79 [id=15574575df68415693e80eaea9955e92] alter complete (version 1)
I20240503 02:09:17.403105 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaaf307b73e46b6a5548a0acba68ba9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.403162 27582 raft_consensus.cc:2454] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.403268 27386 leader_election.cc:302] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.403434 29643 consensus_queue.cc:1035] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.403554 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaaf307b73e46b6a5548a0acba68ba9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.403617 27722 raft_consensus.cc:2454] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.403625 28767 raft_consensus.cc:2781] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.403659 28767 raft_consensus.cc:482] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.403676 28767 raft_consensus.cc:3037] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.404256 28767 raft_consensus.cc:504] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.404369 28767 leader_election.cc:288] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.404577 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaaf307b73e46b6a5548a0acba68ba9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.404635 27583 raft_consensus.cc:3037] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.405275 27583 raft_consensus.cc:2454] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.405658 27386 leader_election.cc:302] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.405754 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaaf307b73e46b6a5548a0acba68ba9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.405812 27718 raft_consensus.cc:3037] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.406363 27718 raft_consensus.cc:2454] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.406749 28767 raft_consensus.cc:2781] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.406802 28767 raft_consensus.cc:686] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.406843 28767 consensus_queue.cc:237] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.408131 27197 catalog_manager.cc:5484] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.413764 28664 tablet_bootstrap.cc:492] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.413811 28664 ts_tablet_manager.cc:1367] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.047s user 0.000s sys 0.000s
I20240503 02:09:17.413937 28664 raft_consensus.cc:348] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.413985 28664 raft_consensus.cc:374] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.414000 28664 raft_consensus.cc:729] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.414033 28664 consensus_queue.cc:260] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.414119 28664 ts_tablet_manager.cc:1398] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.414202 28664 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.414570 28664 tablet_bootstrap.cc:654] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.415419 28664 tablet_bootstrap.cc:492] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.415463 28664 ts_tablet_manager.cc:1367] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.415588 28664 raft_consensus.cc:348] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.415633 28664 raft_consensus.cc:374] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.415650 28664 raft_consensus.cc:729] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.415688 28664 consensus_queue.cc:260] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.415774 28664 ts_tablet_manager.cc:1398] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.415867 28664 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.416196 28664 tablet_bootstrap.cc:654] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.416632 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
W20240503 02:09:17.416626 29925 consensus_queue.cc:1159] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20240503 02:09:17.416745 28664 tablet_bootstrap.cc:492] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.416795 28664 ts_tablet_manager.cc:1367] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.416927 28664 raft_consensus.cc:348] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.416986 28664 raft_consensus.cc:374] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.417013 28664 raft_consensus.cc:729] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.417047 28664 consensus_queue.cc:260] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.417135 28664 ts_tablet_manager.cc:1398] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.417160 28654 ts_tablet_manager.cc:1875] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.417191 28664 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.417441 28655 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.417480 28655 ts_tablet_manager.cc:1367] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.000s
I20240503 02:09:17.417548 28664 tablet_bootstrap.cc:654] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.417596 28655 raft_consensus.cc:348] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.417645 28655 raft_consensus.cc:374] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.417661 28655 raft_consensus.cc:729] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.417699 28655 consensus_queue.cc:260] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.417783 28655 ts_tablet_manager.cc:1398] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.417847 28655 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.418294 28655 tablet_bootstrap.cc:654] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.418510 28654 ts_tablet_manager.cc:1888] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.418550 28654 log.cc:1196] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/949b2d82e23340d3b4ce56934a4db493
I20240503 02:09:17.418725 28654 ts_tablet_manager.cc:1909] T 949b2d82e23340d3b4ce56934a4db493 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.418906 28655 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.418943 28655 ts_tablet_manager.cc:1367] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.419059 28655 raft_consensus.cc:348] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.419103 28655 raft_consensus.cc:374] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.419118 28655 raft_consensus.cc:729] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.419152 28655 consensus_queue.cc:260] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.419246 28655 ts_tablet_manager.cc:1398] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.419551 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.419677 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 949b2d82e23340d3b4ce56934a4db493 (table default.Table_ed36149ca1e94fca90786efbe99c4cba [id=82ad0f3834bc4e9a8740281c5bf54384]) successfully deleted
I20240503 02:09:17.421588 29241 consensus_queue.cc:1035] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.422667 28664 tablet_bootstrap.cc:492] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.422713 28664 ts_tablet_manager.cc:1367] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:17.422834 28664 raft_consensus.cc:348] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.422886 28664 raft_consensus.cc:374] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.422904 28664 raft_consensus.cc:729] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.422952 28664 consensus_queue.cc:260] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.423036 28664 ts_tablet_manager.cc:1398] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.423081 28664 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.423079 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5cd03ad74f1f45fb81b1d87830a025d2" } modify_external_catalogs: true
I20240503 02:09:17.423141 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5cd03ad74f1f45fb81b1d87830a025d2" } modify_external_catalogs: true
I20240503 02:09:17.423408 28664 tablet_bootstrap.cc:654] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.424115 28507 raft_consensus.cc:482] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.424165 28507 raft_consensus.cc:504] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.424263 28507 leader_election.cc:288] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.424432 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ffc308a3b44eed8dce0a99d8f0a6fa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.424443 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ffc308a3b44eed8dce0a99d8f0a6fa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.424490 27718 raft_consensus.cc:2454] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.424489 27451 raft_consensus.cc:2454] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.424630 27524 leader_election.cc:302] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.424721 28507 raft_consensus.cc:2781] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.424762 28507 raft_consensus.cc:482] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.424788 28507 raft_consensus.cc:3037] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.425453 28507 raft_consensus.cc:504] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.425534 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 7d41e4e82931437e808ac39826c285b1
I20240503 02:09:17.425591 28507 leader_election.cc:288] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.425755 28507 raft_consensus.cc:482] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.425801 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d41e4e82931437e808ac39826c285b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.425880 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d41e4e82931437e808ac39826c285b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.425925 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ffc308a3b44eed8dce0a99d8f0a6fa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.425979 27451 raft_consensus.cc:3037] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.425987 28653 tablet_replica.cc:330] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.426069 28653 raft_consensus.cc:2229] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.426102 28653 raft_consensus.cc:2258] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.426213 28664 tablet_bootstrap.cc:492] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.426247 28664 ts_tablet_manager.cc:1367] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:17.426366 28664 raft_consensus.cc:348] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.426412 28664 raft_consensus.cc:374] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.426429 28664 raft_consensus.cc:729] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.426456 28664 consensus_queue.cc:260] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.426645 27451 raft_consensus.cc:2454] T 04ffc308a3b44eed8dce0a99d8f0a6fa P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.426859 27524 leader_election.cc:302] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.426574 28664 ts_tablet_manager.cc:1398] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.426944 29241 raft_consensus.cc:2781] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.426833 27770 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.426978 29241 raft_consensus.cc:686] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.427014 29241 consensus_queue.cc:237] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.427628 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d41e4e82931437e808ac39826c285b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.427716 28661 tablet_replica.cc:330] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.427778 28661 raft_consensus.cc:2229] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.427850 28661 raft_consensus.cc:2258] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.428004 27197 catalog_manager.cc:5484] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.428153 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04ffc308a3b44eed8dce0a99d8f0a6fa" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.428201 27720 raft_consensus.cc:3037] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.428385 28661 ts_tablet_manager.cc:1875] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.428537 29909 log.cc:824] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Log is configured to *not* fsync() on all Append() calls
I20240503 02:09:17.428831 27720 raft_consensus.cc:2454] T 04ffc308a3b44eed8dce0a99d8f0a6fa P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.429687 28661 ts_tablet_manager.cc:1888] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.429736 28661 log.cc:1196] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/7d41e4e82931437e808ac39826c285b1
I20240503 02:09:17.429931 28661 ts_tablet_manager.cc:1909] T 7d41e4e82931437e808ac39826c285b1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.431221 29241 consensus_queue.cc:1035] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.425808 28507 raft_consensus.cc:504] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.431407 27848 consensus_queue.cc:1035] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.431488 28507 leader_election.cc:288] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.431715 28654 tablet_replica.cc:330] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.431775 28654 raft_consensus.cc:2229] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.431820 28654 raft_consensus.cc:2258] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.431813 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58311afeda3d408f9a387c6ec2398e3c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.431864 27718 raft_consensus.cc:2454] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.431885 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58311afeda3d408f9a387c6ec2398e3c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.431934 27450 raft_consensus.cc:2454] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.432039 27524 leader_election.cc:302] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.432515 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_11456da080574c88b77b8871e40fcfdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.432646 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11456da080574c88b77b8871e40fcfdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.434140 28507 raft_consensus.cc:2781] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.434185 28507 raft_consensus.cc:482] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.434206 28507 raft_consensus.cc:3037] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.434819 28507 raft_consensus.cc:504] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.434939 28507 leader_election.cc:288] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.435060 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58311afeda3d408f9a387c6ec2398e3c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.435123 27450 raft_consensus.cc:3037] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.435753 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.435778 27450 raft_consensus.cc:2454] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.435868 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609e5f3b76ac419ea61ae8a2ac42d22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.435932 27524 leader_election.cc:302] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.436161 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 7d41e4e82931437e808ac39826c285b1 (table default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]) successfully deleted
I20240503 02:09:17.436232 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.436303 28654 ts_tablet_manager.cc:1875] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.436347 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58311afeda3d408f9a387c6ec2398e3c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.436393 27718 raft_consensus.cc:3037] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.436967 27718 raft_consensus.cc:2454] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.437090 28655 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.437484 28655 tablet_bootstrap.cc:654] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.437644 28654 ts_tablet_manager.cc:1888] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.437680 28654 log.cc:1196] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/7d41e4e82931437e808ac39826c285b1
I20240503 02:09:17.437867 28654 ts_tablet_manager.cc:1909] T 7d41e4e82931437e808ac39826c285b1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.438026 28655 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.438063 28655 ts_tablet_manager.cc:1367] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.438094 28507 raft_consensus.cc:2781] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.438136 28507 raft_consensus.cc:686] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.438170 28507 consensus_queue.cc:237] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.438206 28655 raft_consensus.cc:348] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.438254 28655 raft_consensus.cc:374] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.438270 28655 raft_consensus.cc:729] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.438314 28655 consensus_queue.cc:260] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.438398 28655 ts_tablet_manager.cc:1398] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.438907 27197 catalog_manager.cc:5484] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.440465 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 7d41e4e82931437e808ac39826c285b1 (table default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]) successfully deleted
I20240503 02:09:17.440659 28653 ts_tablet_manager.cc:1875] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.441898 28653 ts_tablet_manager.cc:1888] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.441938 28653 log.cc:1196] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/7d41e4e82931437e808ac39826c285b1
I20240503 02:09:17.442095 28653 ts_tablet_manager.cc:1909] T 7d41e4e82931437e808ac39826c285b1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.442551 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 7d41e4e82931437e808ac39826c285b1 (table default.Table_5cd03ad74f1f45fb81b1d87830a025d2 [id=f18a872a448e4386b2b6e4c9e13a007d]) successfully deleted
I20240503 02:09:17.442557 28767 raft_consensus.cc:482] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.442592 28767 raft_consensus.cc:504] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.442698 28767 leader_election.cc:288] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.442836 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b8f4c16c99410286f69fef3236e920" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.442889 27718 raft_consensus.cc:2454] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.443006 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b8f4c16c99410286f69fef3236e920" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.443063 27584 raft_consensus.cc:2454] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.443157 27388 leader_election.cc:302] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.443228 28767 raft_consensus.cc:2781] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.443251 28767 raft_consensus.cc:482] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.443269 28767 raft_consensus.cc:3037] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.443660 28767 raft_consensus.cc:504] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.443743 28767 leader_election.cc:288] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.443912 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b8f4c16c99410286f69fef3236e920" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.443915 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53b8f4c16c99410286f69fef3236e920" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.443966 27584 raft_consensus.cc:3037] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.443979 27718 raft_consensus.cc:3037] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.443991 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609e5f3b76ac419ea61ae8a2ac42d22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.444515 27584 raft_consensus.cc:2454] T 53b8f4c16c99410286f69fef3236e920 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.444515 27718 raft_consensus.cc:2454] T 53b8f4c16c99410286f69fef3236e920 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.444648 27386 leader_election.cc:302] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.445199 28664 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.445219 27848 consensus_queue.cc:1035] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.445236 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.445415 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 609e5f3b76ac419ea61ae8a2ac42d22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.445672 28664 tablet_bootstrap.cc:654] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.446182 28767 raft_consensus.cc:2781] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.446229 28767 raft_consensus.cc:686] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.446262 28767 consensus_queue.cc:237] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.447919 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ddee9f12244d4eb2b35bd506b46a46a3" } modify_external_catalogs: true
I20240503 02:09:17.447981 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ddee9f12244d4eb2b35bd506b46a46a3" } modify_external_catalogs: true
I20240503 02:09:17.449394 28664 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.449432 28664 ts_tablet_manager.cc:1367] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.449254 27196 catalog_manager.cc:5484] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.449559 28664 raft_consensus.cc:348] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.449609 28664 raft_consensus.cc:374] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.449625 28664 raft_consensus.cc:729] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.449656 28664 consensus_queue.cc:260] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.449735 28664 ts_tablet_manager.cc:1398] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.450816 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 6ce0b8f7896e400c8847800bccdda58f
I20240503 02:09:17.451042 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce0b8f7896e400c8847800bccdda58f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.451117 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce0b8f7896e400c8847800bccdda58f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.451203 28653 tablet_replica.cc:330] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.451263 28653 raft_consensus.cc:2229] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.451308 28653 raft_consensus.cc:2258] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.451397 28653 ts_tablet_manager.cc:1875] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.452543 28653 ts_tablet_manager.cc:1888] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.452574 28653 log.cc:1196] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/6ce0b8f7896e400c8847800bccdda58f
I20240503 02:09:17.452694 28653 ts_tablet_manager.cc:1909] T 6ce0b8f7896e400c8847800bccdda58f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.453075 28767 consensus_queue.cc:1035] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.453105 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_c98bee3c4f314417828d540f8811584a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.453205 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c98bee3c4f314417828d540f8811584a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.454092 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 6ce0b8f7896e400c8847800bccdda58f (table default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d]) successfully deleted
I20240503 02:09:17.455175 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce0b8f7896e400c8847800bccdda58f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.455252 28661 tablet_replica.cc:330] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.455305 28661 raft_consensus.cc:2229] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.455343 28661 raft_consensus.cc:2258] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.455446 28661 ts_tablet_manager.cc:1875] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.456748 28661 ts_tablet_manager.cc:1888] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.456791 28661 log.cc:1196] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/6ce0b8f7896e400c8847800bccdda58f
I20240503 02:09:17.456969 28661 ts_tablet_manager.cc:1909] T 6ce0b8f7896e400c8847800bccdda58f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.457631 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 6ce0b8f7896e400c8847800bccdda58f (table default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d]) successfully deleted
I20240503 02:09:17.458002 28656 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.458329 28656 tablet_bootstrap.cc:654] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.458745 28654 tablet_replica.cc:330] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.458771 28656 tablet_bootstrap.cc:492] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.458825 28654 raft_consensus.cc:2229] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.458901 28654 raft_consensus.cc:2258] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.458999 28654 ts_tablet_manager.cc:1875] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.458834 28656 ts_tablet_manager.cc:1367] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.459283 28656 raft_consensus.cc:348] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.459352 28656 raft_consensus.cc:374] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.459338 28766 consensus_queue.cc:1035] T b715e519091b49858b528eb83bf33875 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.459379 28656 raft_consensus.cc:729] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.459424 28656 consensus_queue.cc:260] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.459519 28656 ts_tablet_manager.cc:1398] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.460188 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 32d2fdc79fec4c60803eb4f9edb792e5 (DEFAULT_TABLE table=default.Table_c98bee3c4f314417828d540f8811584a [id=84c72b46d87a4eb7a8ca239c5e62534f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.460230 28654 ts_tablet_manager.cc:1888] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.460273 28654 log.cc:1196] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/6ce0b8f7896e400c8847800bccdda58f
I20240503 02:09:17.460309 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d2fdc79fec4c60803eb4f9edb792e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.460456 28654 ts_tablet_manager.cc:1909] T 6ce0b8f7896e400c8847800bccdda58f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.461418 28664 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.461786 28664 tablet_bootstrap.cc:654] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.461946 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 32d2fdc79fec4c60803eb4f9edb792e5 (DEFAULT_TABLE table=default.Table_c98bee3c4f314417828d540f8811584a [id=84c72b46d87a4eb7a8ca239c5e62534f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.462038 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d2fdc79fec4c60803eb4f9edb792e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.462061 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 6ce0b8f7896e400c8847800bccdda58f (table default.Table_ddee9f12244d4eb2b35bd506b46a46a3 [id=6e100540683040cfa172ba132e64928d]) successfully deleted
I20240503 02:09:17.462275 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 32d2fdc79fec4c60803eb4f9edb792e5 (DEFAULT_TABLE table=default.Table_c98bee3c4f314417828d540f8811584a [id=84c72b46d87a4eb7a8ca239c5e62534f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.462257 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_27812ecd4ae84078a50ec3f91617b75f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.462344 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27812ecd4ae84078a50ec3f91617b75f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.462379 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32d2fdc79fec4c60803eb4f9edb792e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.462548 28766 raft_consensus.cc:482] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.462586 28766 raft_consensus.cc:504] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.462675 28766 leader_election.cc:288] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.462844 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bbe30f418e4eafaa881e39a45c3dd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.462903 27583 raft_consensus.cc:2454] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.462877 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bbe30f418e4eafaa881e39a45c3dd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.462934 27720 raft_consensus.cc:2454] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.463128 27386 leader_election.cc:302] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.463160 28655 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.463249 28685 raft_consensus.cc:2781] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.463277 28685 raft_consensus.cc:482] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.463294 28685 raft_consensus.cc:3037] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.463445 28656 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.463513 28655 tablet_bootstrap.cc:654] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.463752 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0dad048e714d40d3ac909246ebf3c49f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.463842 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0dad048e714d40d3ac909246ebf3c49f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.463863 28656 tablet_bootstrap.cc:654] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.463891 28685 raft_consensus.cc:504] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.463985 28685 leader_election.cc:288] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.464418 28664 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.464460 28664 ts_tablet_manager.cc:1367] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.464591 28664 raft_consensus.cc:348] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.464643 28664 raft_consensus.cc:374] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.464660 28664 raft_consensus.cc:729] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.464654 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bbe30f418e4eafaa881e39a45c3dd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.464695 28664 consensus_queue.cc:260] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.464713 27583 raft_consensus.cc:3037] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.464778 28664 ts_tablet_manager.cc:1398] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.464833 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66bbe30f418e4eafaa881e39a45c3dd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.464865 27718 raft_consensus.cc:3037] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.465317 27583 raft_consensus.cc:2454] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.465376 27718 raft_consensus.cc:2454] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.465497 27386 leader_election.cc:302] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.465672 28766 raft_consensus.cc:2781] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.465710 28766 raft_consensus.cc:686] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.465744 28766 consensus_queue.cc:237] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.466467 28656 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.466514 28656 ts_tablet_manager.cc:1367] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.466634 28656 raft_consensus.cc:348] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.466684 28656 raft_consensus.cc:374] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.466701 28656 raft_consensus.cc:729] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.466732 28656 consensus_queue.cc:260] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.466812 28656 ts_tablet_manager.cc:1398] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.468195 27427 tablet_service.cc:1467] Processing CreateTablet for tablet c5707b69d91141ddbc3e9e8d8c3af477 (DEFAULT_TABLE table=default.Table_27812ecd4ae84078a50ec3f91617b75f [id=b4f3301a617d4f00829467fb8f6a390b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.468209 27560 tablet_service.cc:1467] Processing CreateTablet for tablet c5707b69d91141ddbc3e9e8d8c3af477 (DEFAULT_TABLE table=default.Table_27812ecd4ae84078a50ec3f91617b75f [id=b4f3301a617d4f00829467fb8f6a390b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.468250 28655 tablet_bootstrap.cc:492] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.468282 28655 ts_tablet_manager.cc:1367] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:17.468292 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5707b69d91141ddbc3e9e8d8c3af477. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.468292 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5707b69d91141ddbc3e9e8d8c3af477. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.468410 28655 raft_consensus.cc:348] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.468459 28655 raft_consensus.cc:374] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.468475 28655 raft_consensus.cc:729] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.468519 28655 consensus_queue.cc:260] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.468600 28655 ts_tablet_manager.cc:1398] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.468802 27702 tablet_service.cc:1467] Processing CreateTablet for tablet c5707b69d91141ddbc3e9e8d8c3af477 (DEFAULT_TABLE table=default.Table_27812ecd4ae84078a50ec3f91617b75f [id=b4f3301a617d4f00829467fb8f6a390b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.468880 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5707b69d91141ddbc3e9e8d8c3af477. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.469276 28656 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.469384 28655 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.469640 28656 tablet_bootstrap.cc:654] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.469717 28655 tablet_bootstrap.cc:654] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.469951 28664 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.470319 28664 tablet_bootstrap.cc:654] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.471094 29643 raft_consensus.cc:482] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.471134 29643 raft_consensus.cc:504] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.471210 29643 leader_election.cc:288] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.471429 27427 tablet_service.cc:1467] Processing CreateTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.471544 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56dc81ff4644d18a0105722f2f2f1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.471556 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_87499d9c959f4f4b9af432d5321b1b62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.471637 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87499d9c959f4f4b9af432d5321b1b62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.471706 28664 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.471742 28664 ts_tablet_manager.cc:1367] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:17.471859 28664 raft_consensus.cc:348] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.471908 28664 raft_consensus.cc:374] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.471926 28664 raft_consensus.cc:729] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.471977 28664 consensus_queue.cc:260] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.471997 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7282be27904d48f1af33a47693387a5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.472055 27453 raft_consensus.cc:2454] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.472059 28664 ts_tablet_manager.cc:1398] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.472575 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7282be27904d48f1af33a47693387a5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.472635 27584 raft_consensus.cc:2454] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.472657 28655 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.472710 28655 ts_tablet_manager.cc:1367] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.472846 28655 raft_consensus.cc:348] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.472903 28655 raft_consensus.cc:374] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.472921 28655 raft_consensus.cc:729] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.472931 27656 leader_election.cc:302] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.472904 27196 catalog_manager.cc:5484] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.472957 28655 consensus_queue.cc:260] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.472990 27196 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.472992 27853 raft_consensus.cc:2781] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.473027 27853 raft_consensus.cc:482] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.473037 28655 ts_tablet_manager.cc:1398] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.473049 27853 raft_consensus.cc:3037] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.473080 28655 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.473450 28655 tablet_bootstrap.cc:654] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.473696 27853 raft_consensus.cc:504] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.473821 27853 leader_election.cc:288] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.473919 27702 tablet_service.cc:1467] Processing CreateTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.474011 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56dc81ff4644d18a0105722f2f2f1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.474174 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7282be27904d48f1af33a47693387a5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.474225 27584 raft_consensus.cc:3037] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.474639 28656 tablet_bootstrap.cc:492] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.474678 28656 ts_tablet_manager.cc:1367] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:17.474793 27584 raft_consensus.cc:2454] T 7282be27904d48f1af33a47693387a5d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.474797 28656 raft_consensus.cc:348] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.474887 28656 raft_consensus.cc:374] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.474907 28656 raft_consensus.cc:729] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.474942 28656 consensus_queue.cc:260] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.475028 28656 ts_tablet_manager.cc:1398] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.475020 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7282be27904d48f1af33a47693387a5d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.475069 27451 raft_consensus.cc:3037] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.475075 28664 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.475430 28664 tablet_bootstrap.cc:654] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.475692 27451 raft_consensus.cc:2454] T 7282be27904d48f1af33a47693387a5d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.476383 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.476514 27656 leader_election.cc:302] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.476606 28774 raft_consensus.cc:2781] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.476639 28774 raft_consensus.cc:686] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.476670 28774 consensus_queue.cc:237] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.477449 28655 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.477485 28655 ts_tablet_manager.cc:1367] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:17.477619 28655 raft_consensus.cc:348] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.477669 28655 raft_consensus.cc:374] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.477687 28655 raft_consensus.cc:729] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.477728 28655 consensus_queue.cc:260] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.477818 28655 ts_tablet_manager.cc:1398] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.478108 28685 consensus_queue.cc:1035] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.478164 27853 raft_consensus.cc:482] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.478206 27853 raft_consensus.cc:504] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.478303 27853 leader_election.cc:288] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.478421 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5924b7e1ff81434885296e6e3ed6757b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.478485 27583 raft_consensus.cc:2454] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.478623 27656 leader_election.cc:302] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.478876 27853 raft_consensus.cc:2781] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.478911 27853 raft_consensus.cc:482] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.478926 27853 raft_consensus.cc:3037] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.479171 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5924b7e1ff81434885296e6e3ed6757b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.479224 27453 raft_consensus.cc:2454] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.479476 27853 raft_consensus.cc:504] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.479590 27853 leader_election.cc:288] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.480103 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5924b7e1ff81434885296e6e3ed6757b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.480140 27853 raft_consensus.cc:482] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.480170 27583 raft_consensus.cc:3037] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.480182 27853 raft_consensus.cc:504] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.480265 27853 leader_election.cc:288] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.480366 27560 tablet_service.cc:1467] Processing CreateTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.480453 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56dc81ff4644d18a0105722f2f2f1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.480767 27583 raft_consensus.cc:2454] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.481019 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b4b67bcbc8de4b17a042600c659d7c34" } new_table_name: "default.Table_dfed422989eb4f6684bf55a5ec489606" modify_external_catalogs: true
I20240503 02:09:17.481097 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5c7eed47a0be43d4a4cd133971c8cec3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.481225 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5c7eed47a0be43d4a4cd133971c8cec3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.481379 27656 leader_election.cc:302] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.481485 29643 raft_consensus.cc:2781] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.481472 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3a56f2270ce64abe9a8cf5cccfbbf234"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.481559 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a56f2270ce64abe9a8cf5cccfbbf234 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.481595 28656 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.481842 29643 raft_consensus.cc:686] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.481914 28656 tablet_bootstrap.cc:654] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.481905 29643 consensus_queue.cc:237] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.482055 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5924b7e1ff81434885296e6e3ed6757b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.482107 27453 raft_consensus.cc:3037] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.482421 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.482478 27583 raft_consensus.cc:2379] T 640547cbbff2492489dff09073733982 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c2be543b8aad4e80a58d6b4ab669faae in current term 1: Already voted for candidate bd4b79bd8b57443faa6fa5e464570188 in this term.
I20240503 02:09:17.482599 28767 consensus_queue.cc:1035] T d3de7994307949ed9eee04c19f9080e3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.482705 27453 raft_consensus.cc:2454] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.483162 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "640547cbbff2492489dff09073733982" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.483333 27659 leader_election.cc:302] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.483476 28693 raft_consensus.cc:3037] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.484102 28693 raft_consensus.cc:2726] T 640547cbbff2492489dff09073733982 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.485800 28656 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.485841 28656 ts_tablet_manager.cc:1367] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.485936 28664 tablet_bootstrap.cc:492] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.485976 28664 ts_tablet_manager.cc:1367] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:17.485968 28656 raft_consensus.cc:348] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.486024 28656 raft_consensus.cc:374] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.486042 28656 raft_consensus.cc:729] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.486075 28656 consensus_queue.cc:260] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.486121 28664 raft_consensus.cc:348] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.486173 28664 raft_consensus.cc:374] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.486172 28656 ts_tablet_manager.cc:1398] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.486196 28664 raft_consensus.cc:729] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.486227 28664 consensus_queue.cc:260] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.486315 28664 ts_tablet_manager.cc:1398] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.487253 27770 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.487838 27500 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.489424 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 389839158b314fd7a34852af7b4afadc (DEFAULT_TABLE table=default.Table_87499d9c959f4f4b9af432d5321b1b62 [id=66944c8b652d4974b1cfaec358ac5fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.489523 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389839158b314fd7a34852af7b4afadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.490729 28656 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.491096 28656 tablet_bootstrap.cc:654] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.491433 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 389839158b314fd7a34852af7b4afadc (DEFAULT_TABLE table=default.Table_87499d9c959f4f4b9af432d5321b1b62 [id=66944c8b652d4974b1cfaec358ac5fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.491531 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389839158b314fd7a34852af7b4afadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.491947 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 389839158b314fd7a34852af7b4afadc (DEFAULT_TABLE table=default.Table_87499d9c959f4f4b9af432d5321b1b62 [id=66944c8b652d4974b1cfaec358ac5fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.492039 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 389839158b314fd7a34852af7b4afadc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.492658 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 99265e78c1d74facb95bb8f7abf71110 (DEFAULT_TABLE table=default.Table_3a56f2270ce64abe9a8cf5cccfbbf234 [id=8200ea16349c4080a0634fa6a89029f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.492658 29566 tablet.cc:1722] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.492758 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99265e78c1d74facb95bb8f7abf71110. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.492800 27200 catalog_manager.cc:5484] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.492902 27200 catalog_manager.cc:5484] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.493062 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 99265e78c1d74facb95bb8f7abf71110 (DEFAULT_TABLE table=default.Table_3a56f2270ce64abe9a8cf5cccfbbf234 [id=8200ea16349c4080a0634fa6a89029f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.493140 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99265e78c1d74facb95bb8f7abf71110. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.493166 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.493237 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d587597b0e4243ad98a7f20a1043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.493891 29558 tablet.cc:1722] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.494333 28655 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.494347 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.494432 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d587597b0e4243ad98a7f20a1043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.494685 28655 tablet_bootstrap.cc:654] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.494834 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.494925 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d587597b0e4243ad98a7f20a1043f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.495095 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_603f20ef8a6f4c30b375ab68d285265d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.495189 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_603f20ef8a6f4c30b375ab68d285265d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.495275 28710 tablet.cc:1722] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.495599 28664 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.495967 28664 tablet_bootstrap.cc:654] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.497066 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e938bdd853d740349a283669f6a3219d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.497156 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e938bdd853d740349a283669f6a3219d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:17.498474 27197 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:17.500404 27718 raft_consensus.cc:1261] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.500501 28656 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.500541 28656 ts_tablet_manager.cc:1367] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:17.500659 28656 raft_consensus.cc:348] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.500703 28656 raft_consensus.cc:374] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.500720 28656 raft_consensus.cc:729] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.500751 28656 consensus_queue.cc:260] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.500977 28656 ts_tablet_manager.cc:1398] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.501045 28656 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.501052 27583 raft_consensus.cc:1261] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.501344 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_dfed422989eb4f6684bf55a5ec489606 [id=a56811ee47c84a1f9e43ed591962b726] alter complete (version 1)
I20240503 02:09:17.501413 28656 tablet_bootstrap.cc:654] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.502007 28766 consensus_queue.cc:1035] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.502372 28766 consensus_queue.cc:1035] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.502969 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 9001e72388f7452e85476a5d2b3cf1c1 (DEFAULT_TABLE table=default.Table_603f20ef8a6f4c30b375ab68d285265d [id=a01c8a8721544a84a9f2a203c0a54eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.502959 27565 tablet_service.cc:1467] Processing CreateTablet for tablet e51a7970b0e147358dab5f6814502ef0 (DEFAULT_TABLE table=default.Table_e938bdd853d740349a283669f6a3219d [id=2b0b549adfb74c9e9e9c86af21d1a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.503052 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a7970b0e147358dab5f6814502ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.503051 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9001e72388f7452e85476a5d2b3cf1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.503152 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 9001e72388f7452e85476a5d2b3cf1c1 (DEFAULT_TABLE table=default.Table_603f20ef8a6f4c30b375ab68d285265d [id=a01c8a8721544a84a9f2a203c0a54eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.503242 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9001e72388f7452e85476a5d2b3cf1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.504088 27703 tablet_service.cc:1467] Processing CreateTablet for tablet e51a7970b0e147358dab5f6814502ef0 (DEFAULT_TABLE table=default.Table_e938bdd853d740349a283669f6a3219d [id=2b0b549adfb74c9e9e9c86af21d1a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.504088 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 9001e72388f7452e85476a5d2b3cf1c1 (DEFAULT_TABLE table=default.Table_603f20ef8a6f4c30b375ab68d285265d [id=a01c8a8721544a84a9f2a203c0a54eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.504171 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9001e72388f7452e85476a5d2b3cf1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.504170 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a7970b0e147358dab5f6814502ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.504618 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 99265e78c1d74facb95bb8f7abf71110 (DEFAULT_TABLE table=default.Table_3a56f2270ce64abe9a8cf5cccfbbf234 [id=8200ea16349c4080a0634fa6a89029f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.504653 27848 consensus_queue.cc:1035] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.504702 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99265e78c1d74facb95bb8f7abf71110. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.504914 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.505815 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.507601 27425 tablet_service.cc:1467] Processing CreateTablet for tablet e51a7970b0e147358dab5f6814502ef0 (DEFAULT_TABLE table=default.Table_e938bdd853d740349a283669f6a3219d [id=2b0b549adfb74c9e9e9c86af21d1a43a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.507689 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51a7970b0e147358dab5f6814502ef0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.507836 28656 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.507869 28656 ts_tablet_manager.cc:1367] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:17.507984 28656 raft_consensus.cc:348] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.508033 28656 raft_consensus.cc:374] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.508049 28656 raft_consensus.cc:729] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.508085 28656 consensus_queue.cc:260] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.508167 28656 ts_tablet_manager.cc:1398] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.508224 28656 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.508535 28656 tablet_bootstrap.cc:654] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.508673 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap replayed 1/1 log segments. Stats: ops{read=840 overwritten=0 applied=840 ignored=0} inserts{seen=401 ignored=0} mutations{seen=996 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:09:17.508666 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_676914005e65475eac45ad325ccae621" } new_table_name: "default.Table_ddd88b53492640d1b2efc76effd0b7a0" modify_external_catalogs: true
I20240503 02:09:17.508973 29909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: Bootstrap complete.
I20240503 02:09:17.509004 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b44708e810b449d38c560721b1a7d665" } modify_external_catalogs: true
I20240503 02:09:17.509049 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b44708e810b449d38c560721b1a7d665" } modify_external_catalogs: true
I20240503 02:09:17.509857 28655 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.509896 28655 ts_tablet_manager.cc:1367] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240503 02:09:17.510030 28655 raft_consensus.cc:348] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.510030 28710 tablet.cc:1722] T 66bbe30f418e4eafaa881e39a45c3dd7 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.510088 28655 raft_consensus.cc:374] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.510107 28655 raft_consensus.cc:729] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.510138 28655 consensus_queue.cc:260] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.510221 28655 ts_tablet_manager.cc:1398] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.510258 28692 mvcc.cc:205] Tried to move back new op lower bound from 7023420037120397312 to 7023420036979953664. Current Snapshot: MvccSnapshot[applied={T|T < 7023420037120397312}]
I20240503 02:09:17.510459 29558 tablet.cc:1722] T 66bbe30f418e4eafaa881e39a45c3dd7 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.510480 28655 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.510855 28655 tablet_bootstrap.cc:654] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.510802 29909 raft_consensus.cc:348] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:17.511094 29909 raft_consensus.cc:729] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c324c9220fd14d3c817b18bbd3774439, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.511220 29909 consensus_queue.cc:260] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 840, Last appended: 3.840, Last appended by leader: 840, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } }
I20240503 02:09:17.511494 29909 sys_catalog.cc:564] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:09:17.511514 28664 tablet_bootstrap.cc:492] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.511547 28664 ts_tablet_manager.cc:1367] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:17.511683 28664 raft_consensus.cc:348] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.511731 28664 raft_consensus.cc:374] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.511747 28664 raft_consensus.cc:729] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.511790 28664 consensus_queue.cc:260] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.511880 28664 ts_tablet_manager.cc:1398] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.511924 28664 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.512270 28664 tablet_bootstrap.cc:654] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.514050 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 837debf700f74dc680eb88168aeac413
I20240503 02:09:17.514355 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 837debf700f74dc680eb88168aeac413 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.514396 29241 consensus_queue.cc:1035] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.514451 28654 tablet_replica.cc:330] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.514499 28654 raft_consensus.cc:2229] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.514539 28654 raft_consensus.cc:2258] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.514573 27635 heartbeater.cc:347] Connected to a master server at 127.26.117.62:37529
I20240503 02:09:17.515128 28654 ts_tablet_manager.cc:1875] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.515532 28656 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.515566 28656 ts_tablet_manager.cc:1367] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:17.515686 28656 raft_consensus.cc:348] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.515743 28656 raft_consensus.cc:374] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.515761 28656 raft_consensus.cc:729] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.515798 28656 consensus_queue.cc:260] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.515878 28656 ts_tablet_manager.cc:1398] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.515924 28656 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.516261 28656 tablet_bootstrap.cc:654] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.516435 28654 ts_tablet_manager.cc:1888] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.516471 28654 log.cc:1196] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/837debf700f74dc680eb88168aeac413
I20240503 02:09:17.516609 29868 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" instance_seqno: 1714702146863903) as {username='slave'} at 127.26.117.3:45875; Asking this server to re-register.
I20240503 02:09:17.516644 28654 ts_tablet_manager.cc:1909] T 837debf700f74dc680eb88168aeac413 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.516834 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 837debf700f74dc680eb88168aeac413 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.516906 28661 tablet_replica.cc:330] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.516969 28661 raft_consensus.cc:2229] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.517006 28661 raft_consensus.cc:2258] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.517230 28661 ts_tablet_manager.cc:1875] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.517246 28656 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.517282 28656 ts_tablet_manager.cc:1367] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.517292 27770 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:17.517414 28656 raft_consensus.cc:348] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.517463 28656 raft_consensus.cc:374] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.517479 28656 raft_consensus.cc:729] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.517526 28656 consensus_queue.cc:260] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.517602 28656 ts_tablet_manager.cc:1398] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.517648 28656 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.517879 29868 ts_manager.cc:194] Registered new tserver with Master: c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.517957 28656 tablet_bootstrap.cc:654] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.518663 28661 ts_tablet_manager.cc:1888] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.518718 28661 log.cc:1196] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/837debf700f74dc680eb88168aeac413
I20240503 02:09:17.518899 28661 ts_tablet_manager.cc:1909] T 837debf700f74dc680eb88168aeac413 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.519176 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 837debf700f74dc680eb88168aeac413 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.519268 28653 tablet_replica.cc:330] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.519316 28653 raft_consensus.cc:2229] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.519382 28653 raft_consensus.cc:2258] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.519446 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 837debf700f74dc680eb88168aeac413 (table default.Table_b44708e810b449d38c560721b1a7d665 [id=260be253b97d4a5681ac6d6875ad7414]) successfully deleted
I20240503 02:09:17.519675 29868 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" instance_seqno: 1714702146630581) as {username='slave'} at 127.26.117.1:47073; Asking this server to re-register.
I20240503 02:09:17.519832 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 837debf700f74dc680eb88168aeac413 (table default.Table_b44708e810b449d38c560721b1a7d665 [id=260be253b97d4a5681ac6d6875ad7414]) successfully deleted
I20240503 02:09:17.520156 28653 ts_tablet_manager.cc:1875] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.520285 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_83e83f7c5d1049358d61e4f86a74c868"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.520380 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83e83f7c5d1049358d61e4f86a74c868 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.520548 27500 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:17.520591 29991 catalog_manager.cc:1259] Loaded cluster ID: 6c8a1e19e4bd4d46835340a45d759cdd
I20240503 02:09:17.520623 29991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: loading cluster ID for follower catalog manager: success
I20240503 02:09:17.520844 27853 raft_consensus.cc:482] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.520887 27853 raft_consensus.cc:504] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.520995 27853 leader_election.cc:288] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.521106 29868 ts_manager.cc:194] Registered new tserver with Master: bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:17.521103 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609e5f3b76ac419ea61ae8a2ac42d22f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.521162 27584 raft_consensus.cc:2454] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.521143 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609e5f3b76ac419ea61ae8a2ac42d22f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.521195 27449 raft_consensus.cc:2454] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.521266 27656 leader_election.cc:302] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.521376 27853 raft_consensus.cc:2781] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.521405 27853 raft_consensus.cc:482] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.521423 27853 raft_consensus.cc:3037] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.521540 28653 ts_tablet_manager.cc:1888] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.521589 28653 log.cc:1196] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/837debf700f74dc680eb88168aeac413
I20240503 02:09:17.521764 28653 ts_tablet_manager.cc:1909] T 837debf700f74dc680eb88168aeac413 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.522042 27853 raft_consensus.cc:504] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.522142 27853 leader_election.cc:288] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.522308 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609e5f3b76ac419ea61ae8a2ac42d22f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.522341 29868 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" instance_seqno: 1714702146750084) as {username='slave'} at 127.26.117.2:48739; Asking this server to re-register.
I20240503 02:09:17.522362 27584 raft_consensus.cc:3037] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.522910 29566 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.522966 27584 raft_consensus.cc:2454] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.523169 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "609e5f3b76ac419ea61ae8a2ac42d22f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.523223 27451 raft_consensus.cc:3037] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.523499 28710 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.523720 27656 leader_election.cc:302] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.523816 27451 raft_consensus.cc:2454] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.523874 29558 tablet.cc:1722] T d5fb40e254ab40f09cc72093bf50c91b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.524016 27853 raft_consensus.cc:2781] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.524051 27853 raft_consensus.cc:686] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.524080 27853 consensus_queue.cc:237] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.525076 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 837debf700f74dc680eb88168aeac413 (table default.Table_b44708e810b449d38c560721b1a7d665 [id=260be253b97d4a5681ac6d6875ad7414]) successfully deleted
I20240503 02:09:17.525660 29991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: acquiring CA information for follower catalog manager: success
I20240503 02:09:17.526072 28656 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.526113 28656 ts_tablet_manager.cc:1367] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:17.526229 28656 raft_consensus.cc:348] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.526275 28656 raft_consensus.cc:374] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.526290 28656 raft_consensus.cc:729] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.526320 28656 consensus_queue.cc:260] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.526333 27635 heartbeater.cc:464] Registering TS with master...
I20240503 02:09:17.526412 28656 ts_tablet_manager.cc:1398] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.526628 29991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20240503 02:09:17.526938 29868 ts_manager.cc:194] Registered new tserver with Master: 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.527266 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (DEFAULT_TABLE table=default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.527361 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.528677 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (DEFAULT_TABLE table=default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.528770 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.529492 27201 catalog_manager.cc:5484] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.529623 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (DEFAULT_TABLE table=default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.529713 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.530052 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9beb4cf015b3468a9aa10491b78c26da" } new_table_name: "default.Table_04b143bee9ff47b39766ba185154531c" modify_external_catalogs: true
I20240503 02:09:17.531484 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0978045ecff040a09a7a1cd91e12dcb1" } new_table_name: "default.Table_41b9ffbd06cd4e66873d63d6a8b3dc9f" modify_external_catalogs: true
I20240503 02:09:17.532027 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_ddd88b53492640d1b2efc76effd0b7a0 [id=97264f1a599a4a9680a9f284e2633241] alter complete (version 2)
I20240503 02:09:17.532851 28656 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.533201 28656 tablet_bootstrap.cc:654] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.533721 28656 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.533763 28656 ts_tablet_manager.cc:1367] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.533878 28656 raft_consensus.cc:348] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.533947 28656 raft_consensus.cc:374] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.533986 28656 raft_consensus.cc:729] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.534025 28656 consensus_queue.cc:260] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.534117 28656 ts_tablet_manager.cc:1398] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.534222 29558 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.534586 28710 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.537215 29566 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.539355 28664 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.539399 28664 ts_tablet_manager.cc:1367] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20240503 02:09:17.539516 28664 raft_consensus.cc:348] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.539569 28664 raft_consensus.cc:374] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.539584 28664 raft_consensus.cc:729] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.539620 28664 consensus_queue.cc:260] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.539692 28664 ts_tablet_manager.cc:1398] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.539754 28664 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.540123 28664 tablet_bootstrap.cc:654] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.540132 27200 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_04b143bee9ff47b39766ba185154531c [id=53e0fca749ea4dc699450e4503b21c03] alter complete (version 2)
I20240503 02:09:17.540856 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet cab656d127ee405bb11ad94df6eba430 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.543102 29558 tablet.cc:1722] T 766a17e99ae64b878920d451cc814a3c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.543409 28507 raft_consensus.cc:482] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.543466 28507 raft_consensus.cc:504] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.543567 28507 leader_election.cc:288] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.543756 28710 tablet.cc:1722] T 766a17e99ae64b878920d451cc814a3c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.543769 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f42c6ab1499d4b058241376b6f211bde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.543836 27718 raft_consensus.cc:2454] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.543951 27523 leader_election.cc:302] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.544019 28507 raft_consensus.cc:2781] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.544042 28507 raft_consensus.cc:482] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.544059 28507 raft_consensus.cc:3037] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.544626 28655 tablet_bootstrap.cc:492] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.544665 28655 ts_tablet_manager.cc:1367] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.001s
I20240503 02:09:17.544687 28507 raft_consensus.cc:504] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.544781 28507 leader_election.cc:288] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.544801 28655 raft_consensus.cc:348] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.544855 28655 raft_consensus.cc:374] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.544847 29566 tablet.cc:1722] T 66bbe30f418e4eafaa881e39a45c3dd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.544873 28655 raft_consensus.cc:729] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.544907 28655 consensus_queue.cc:260] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.544984 28655 ts_tablet_manager.cc:1398] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.545042 28655 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.545176 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 66bbe30f418e4eafaa881e39a45c3dd7 (table default.Table_ee0946e70ae74889ace95c7aec71f6d2 [id=61076672a8034d1ebb1077d0795360d1]). Expected version 2 got 0
I20240503 02:09:17.545414 28655 tablet_bootstrap.cc:654] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.545683 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f42c6ab1499d4b058241376b6f211bde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.545737 27453 raft_consensus.cc:2454] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.545810 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f42c6ab1499d4b058241376b6f211bde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.545857 27720 raft_consensus.cc:3037] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.546003 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet cab656d127ee405bb11ad94df6eba430
I20240503 02:09:17.546090 27201 catalog_manager.cc:6554] ReplaceTablet: tablet cab656d127ee405bb11ad94df6eba430 deleted and replaced by tablet ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.546471 27720 raft_consensus.cc:2454] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.546630 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet cab656d127ee405bb11ad94df6eba430 with delete_type TABLET_DATA_DELETED (replaced by tablet ab2acfdf916c40878a51a85eaadf7dbf) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.546707 28661 tablet_replica.cc:330] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.546717 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet cab656d127ee405bb11ad94df6eba430 with delete_type TABLET_DATA_DELETED (replaced by tablet ab2acfdf916c40878a51a85eaadf7dbf) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.546751 28661 raft_consensus.cc:2229] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.546782 28661 raft_consensus.cc:2258] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.546811 28653 tablet_replica.cc:330] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.546869 28653 raft_consensus.cc:2229] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.546901 28653 raft_consensus.cc:2258] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.546991 28661 ts_tablet_manager.cc:1875] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.547084 28653 ts_tablet_manager.cc:1875] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.547814 27523 leader_election.cc:302] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.547883 28410 raft_consensus.cc:2781] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.547914 28410 raft_consensus.cc:686] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.547950 28410 consensus_queue.cc:237] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.548292 28661 ts_tablet_manager.cc:1888] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.548292 28653 ts_tablet_manager.cc:1888] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.548336 28661 log.cc:1196] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/cab656d127ee405bb11ad94df6eba430
I20240503 02:09:17.548339 28653 log.cc:1196] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/cab656d127ee405bb11ad94df6eba430
I20240503 02:09:17.548517 28653 ts_tablet_manager.cc:1909] T cab656d127ee405bb11ad94df6eba430 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.548528 28661 ts_tablet_manager.cc:1909] T cab656d127ee405bb11ad94df6eba430 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.548652 29241 raft_consensus.cc:482] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.548696 29241 raft_consensus.cc:504] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.548777 29241 leader_election.cc:288] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.549254 27426 tablet_service.cc:1467] Processing CreateTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.549250 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2079721e8b764ff48a560052fa95adfa" } modify_external_catalogs: true
I20240503 02:09:17.549309 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2079721e8b764ff48a560052fa95adfa" } modify_external_catalogs: true
I20240503 02:09:17.549364 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2acfdf916c40878a51a85eaadf7dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.549379 27562 tablet_service.cc:1467] Processing CreateTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.549469 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2acfdf916c40878a51a85eaadf7dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.549468 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1513d47f8f1848448db5a23d72c63c8a" } new_table_name: "default.Table_b72419469762486eb080ac91f57505df" modify_external_catalogs: true
I20240503 02:09:17.549952 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56dc81ff4644d18a0105722f2f2f1e7" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.550025 27720 raft_consensus.cc:2454] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.550040 27703 tablet_service.cc:1467] Processing CreateTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.550125 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab2acfdf916c40878a51a85eaadf7dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.550132 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet cab656d127ee405bb11ad94df6eba430 with delete_type TABLET_DATA_DELETED (replaced by tablet ab2acfdf916c40878a51a85eaadf7dbf) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.550194 28654 tablet_replica.cc:330] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.550235 28654 raft_consensus.cc:2229] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.550303 28654 raft_consensus.cc:2258] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.550525 28654 ts_tablet_manager.cc:1875] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.550859 28656 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.551195 28656 tablet_bootstrap.cc:654] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.551455 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56dc81ff4644d18a0105722f2f2f1e7" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.551524 27449 raft_consensus.cc:2454] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.551803 28654 ts_tablet_manager.cc:1888] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.551842 28654 log.cc:1196] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/cab656d127ee405bb11ad94df6eba430
I20240503 02:09:17.552026 27523 leader_election.cc:302] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.552043 28654 ts_tablet_manager.cc:1909] T cab656d127ee405bb11ad94df6eba430 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.552119 29241 raft_consensus.cc:2781] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.552142 29241 raft_consensus.cc:482] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.552157 29241 raft_consensus.cc:3037] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.552222 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f42c6ab1499d4b058241376b6f211bde" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.552266 27453 raft_consensus.cc:3037] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.552417 27201 catalog_manager.cc:5484] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.552724 29241 raft_consensus.cc:504] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.552821 29241 leader_election.cc:288] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.552824 27453 raft_consensus.cc:2454] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.553258 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet cab656d127ee405bb11ad94df6eba430 (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
I20240503 02:09:17.553606 29241 consensus_queue.cc:1035] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.553656 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56dc81ff4644d18a0105722f2f2f1e7" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.553704 27720 raft_consensus.cc:3037] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.554270 27720 raft_consensus.cc:2454] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.554368 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet cab656d127ee405bb11ad94df6eba430 (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
I20240503 02:09:17.554422 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56dc81ff4644d18a0105722f2f2f1e7" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.554471 27453 raft_consensus.cc:3037] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.554963 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 811e1aa52cb2458792359e9baebd5e5c
I20240503 02:09:17.554998 27523 leader_election.cc:302] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.555017 27453 raft_consensus.cc:2454] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.555068 29241 raft_consensus.cc:2781] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.555097 29241 raft_consensus.cc:686] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.555127 29241 consensus_queue.cc:237] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.555236 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 811e1aa52cb2458792359e9baebd5e5c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.556133 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 811e1aa52cb2458792359e9baebd5e5c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.556278 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_83e83f7c5d1049358d61e4f86a74c868" } modify_external_catalogs: true
I20240503 02:09:17.556339 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_83e83f7c5d1049358d61e4f86a74c868" } modify_external_catalogs: true
I20240503 02:09:17.556342 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 811e1aa52cb2458792359e9baebd5e5c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.556416 28654 tablet_replica.cc:330] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.556463 28654 raft_consensus.cc:2229] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.556494 28654 raft_consensus.cc:2258] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.556598 28654 ts_tablet_manager.cc:1875] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.557233 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 24e32edb5ccf44199fce2ec6a2c8cf63 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.557772 28654 ts_tablet_manager.cc:1888] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.557812 28654 log.cc:1196] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/811e1aa52cb2458792359e9baebd5e5c
I20240503 02:09:17.557861 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_41b9ffbd06cd4e66873d63d6a8b3dc9f [id=a876b3d23f9b4ec2821cd873c06fe303] alter complete (version 1)
I20240503 02:09:17.557986 28654 ts_tablet_manager.cc:1909] T 811e1aa52cb2458792359e9baebd5e5c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.558311 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 811e1aa52cb2458792359e9baebd5e5c (table default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8]) successfully deleted
I20240503 02:09:17.558745 27718 raft_consensus.cc:1261] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.558869 28661 tablet_replica.cc:330] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.558928 28661 raft_consensus.cc:2229] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.558957 28661 raft_consensus.cc:2258] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.559063 28661 ts_tablet_manager.cc:1875] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.560024 28653 tablet_replica.cc:330] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.559999 28410 consensus_queue.cc:1035] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.560089 28653 raft_consensus.cc:2229] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.560137 28653 raft_consensus.cc:2258] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.560225 28653 ts_tablet_manager.cc:1875] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.560279 28661 ts_tablet_manager.cc:1888] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.560315 28661 log.cc:1196] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/811e1aa52cb2458792359e9baebd5e5c
I20240503 02:09:17.560374 27198 catalog_manager.cc:5484] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.560477 28661 ts_tablet_manager.cc:1909] T 811e1aa52cb2458792359e9baebd5e5c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.561084 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 24e32edb5ccf44199fce2ec6a2c8cf63
I20240503 02:09:17.561165 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 24e32edb5ccf44199fce2ec6a2c8cf63 deleted and replaced by tablet 4d471aedbe2a45369909f9590e3dfb83
I20240503 02:09:17.561260 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 811e1aa52cb2458792359e9baebd5e5c (table default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8]) successfully deleted
I20240503 02:09:17.561371 28653 ts_tablet_manager.cc:1888] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.561435 28653 log.cc:1196] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/811e1aa52cb2458792359e9baebd5e5c
I20240503 02:09:17.561669 28653 ts_tablet_manager.cc:1909] T 811e1aa52cb2458792359e9baebd5e5c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.561911 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d471aedbe2a45369909f9590e3dfb83) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.562296 28767 consensus_queue.cc:1035] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.562378 28661 tablet_replica.cc:330] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.562436 28661 raft_consensus.cc:2229] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.562476 28661 raft_consensus.cc:2258] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.562741 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d471aedbe2a45369909f9590e3dfb83) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.562738 27449 raft_consensus.cc:1261] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.562877 28661 ts_tablet_manager.cc:1875] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.563315 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet cab656d127ee405bb11ad94df6eba430 (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
I20240503 02:09:17.563369 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 811e1aa52cb2458792359e9baebd5e5c (table default.Table_2079721e8b764ff48a560052fa95adfa [id=076a99c72f8645e59fbc62a96e3887b8]) successfully deleted
I20240503 02:09:17.563443 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e32edb5ccf44199fce2ec6a2c8cf63 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d471aedbe2a45369909f9590e3dfb83) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.563509 28654 tablet_replica.cc:330] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.563555 28654 raft_consensus.cc:2229] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.563591 28654 raft_consensus.cc:2258] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.563791 28654 ts_tablet_manager.cc:1875] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.564235 28661 ts_tablet_manager.cc:1888] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.564292 28661 log.cc:1196] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/24e32edb5ccf44199fce2ec6a2c8cf63
I20240503 02:09:17.564466 28661 ts_tablet_manager.cc:1909] T 24e32edb5ccf44199fce2ec6a2c8cf63 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.564944 28654 ts_tablet_manager.cc:1888] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.564939 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:17.564992 28654 log.cc:1196] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/24e32edb5ccf44199fce2ec6a2c8cf63
I20240503 02:09:17.565166 28654 ts_tablet_manager.cc:1909] T 24e32edb5ccf44199fce2ec6a2c8cf63 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.565181 28656 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.565222 28656 ts_tablet_manager.cc:1367] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.000s
I20240503 02:09:17.565547 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:17.565699 28410 consensus_queue.cc:1035] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.565995 28656 raft_consensus.cc:348] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.566057 28656 raft_consensus.cc:374] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.566076 28656 raft_consensus.cc:729] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.566102 28656 consensus_queue.cc:260] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.566190 28656 ts_tablet_manager.cc:1398] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.568199 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 6dcd0014c93f4a3296e33c7d3221e06e (table default.Table_b72419469762486eb080ac91f57505df [id=e2143b8ffeae471d93521e4896bb63a3]). Expected version 1 got 0
I20240503 02:09:17.568408 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 37ed1b8a0ae9439a8a7f8824eb68e01e
I20240503 02:09:17.568670 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.568745 28654 tablet_replica.cc:330] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.568809 28654 raft_consensus.cc:2229] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.568837 28654 raft_consensus.cc:2258] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.568950 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.569564 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 6dcd0014c93f4a3296e33c7d3221e06e (table default.Table_b72419469762486eb080ac91f57505df [id=e2143b8ffeae471d93521e4896bb63a3]). Expected version 1 got 0
I20240503 02:09:17.569550 28507 consensus_queue.cc:1035] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.570880 28507 raft_consensus.cc:482] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.570928 28507 raft_consensus.cc:504] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.571156 28507 leader_election.cc:288] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.571343 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.571414 27718 raft_consensus.cc:2454] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.571614 27523 leader_election.cc:302] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.571700 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.571795 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d471aedbe2a45369909f9590e3dfb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.571820 28507 raft_consensus.cc:2781] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.571846 28507 raft_consensus.cc:482] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.571862 28507 raft_consensus.cc:3037] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.573004 28507 raft_consensus.cc:504] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.573117 28507 leader_election.cc:288] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.573278 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.573451 27718 raft_consensus.cc:3037] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.573902 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.573987 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 8a5b3f2a01b5453983b0d767ff376a6c from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.574223 27718 raft_consensus.cc:2454] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.574385 27523 leader_election.cc:302] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.574484 29240 raft_consensus.cc:2781] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.574541 29240 raft_consensus.cc:686] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.574575 29240 consensus_queue.cc:237] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:17.574916 27189 catalog_manager.cc:4810] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): delete failed for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37ed1b8a0ae9439a8a7f8824eb68e01e already in progress: creating tablet
I20240503 02:09:17.575088 28654 ts_tablet_manager.cc:1875] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.576236 28654 ts_tablet_manager.cc:1888] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.576290 28654 log.cc:1196] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/37ed1b8a0ae9439a8a7f8824eb68e01e
I20240503 02:09:17.576463 28654 ts_tablet_manager.cc:1909] T 37ed1b8a0ae9439a8a7f8824eb68e01e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.576848 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (table default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]) successfully deleted
I20240503 02:09:17.579104 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet abaaf307b73e46b6a5548a0acba68ba9 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.581947 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet a3650180caf546878779d1c89950b2c1 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.584342 29643 raft_consensus.cc:482] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.584404 29643 raft_consensus.cc:504] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.584506 29643 leader_election.cc:288] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.584554 28767 raft_consensus.cc:482] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.584595 28767 raft_consensus.cc:504] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.584622 29643 consensus_queue.cc:1035] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.584683 28767 leader_election.cc:288] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.584749 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.584789 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.584842 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d471aedbe2a45369909f9590e3dfb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.584848 27451 raft_consensus.cc:2379] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c2be543b8aad4e80a58d6b4ab669faae in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:17.585023 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.585088 27722 raft_consensus.cc:2379] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:17.586395 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.586561 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50566b0702db4f85b581931f415ca42d" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.586932 27386 leader_election.cc:302] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.587129 27656 leader_election.cc:302] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.587163 28409 raft_consensus.cc:3037] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.587255 28693 raft_consensus.cc:3037] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.587875 28409 raft_consensus.cc:2726] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.587905 28693 raft_consensus.cc:2726] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.587975 28656 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.588140 28693 raft_consensus.cc:482] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.588183 28693 raft_consensus.cc:504] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.588212 28685 consensus_queue.cc:1035] T cb958efd36594d5390ae617940aed908 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.588277 28693 leader_election.cc:288] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.588325 28656 tablet_bootstrap.cc:654] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.588766 28653 tablet_replica.cc:330] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.588840 28653 raft_consensus.cc:2229] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.588896 28653 raft_consensus.cc:2258] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.588878 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389839158b314fd7a34852af7b4afadc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.588928 27451 raft_consensus.cc:2454] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.588650 27201 catalog_manager.cc:5484] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.589012 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.589259 28653 ts_tablet_manager.cc:1875] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.589634 29566 tablet.cc:1722] T 766a17e99ae64b878920d451cc814a3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.589823 27659 leader_election.cc:302] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.589974 27853 raft_consensus.cc:2781] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.590010 27853 raft_consensus.cc:482] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.590029 27853 raft_consensus.cc:3037] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.590545 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:17.590621 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:17.590613 27853 raft_consensus.cc:504] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.590714 27853 leader_election.cc:288] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.591042 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389839158b314fd7a34852af7b4afadc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.591099 27451 raft_consensus.cc:3037] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.591348 28653 ts_tablet_manager.cc:1888] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.591390 28653 log.cc:1196] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/24e32edb5ccf44199fce2ec6a2c8cf63
I20240503 02:09:17.591568 28653 ts_tablet_manager.cc:1909] T 24e32edb5ccf44199fce2ec6a2c8cf63 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.591668 27451 raft_consensus.cc:2454] T 389839158b314fd7a34852af7b4afadc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.592067 27659 leader_election.cc:302] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.592139 27853 raft_consensus.cc:2781] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.592175 27853 raft_consensus.cc:686] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.592206 27853 consensus_queue.cc:237] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.592689 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.592722 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.592823 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d471aedbe2a45369909f9590e3dfb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.593279 27450 raft_consensus.cc:2454] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.593482 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb76d46144e74642bdcb0146ed419ce5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.593541 27451 raft_consensus.cc:2362] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:17.593751 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 24e32edb5ccf44199fce2ec6a2c8cf63 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
W20240503 02:09:17.593824 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37ed1b8a0ae9439a8a7f8824eb68e01e already in progress: creating tablet
I20240503 02:09:17.595240 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8a5b3f2a01b5453983b0d767ff376a6c
I20240503 02:09:17.595233 28740 consensus_queue.cc:1035] T 657eaa8008d34178b52af2ee7af49e03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.595311 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 8a5b3f2a01b5453983b0d767ff376a6c deleted and replaced by tablet 2834e40454ef48429c31b50da5d2ab0f
I20240503 02:09:17.595362 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a3650180caf546878779d1c89950b2c1
I20240503 02:09:17.595147 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389839158b314fd7a34852af7b4afadc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.595407 27200 catalog_manager.cc:6554] ReplaceTablet: tablet a3650180caf546878779d1c89950b2c1 deleted and replaced by tablet 31b262a1decf457ebab85ea106273fe2
I20240503 02:09:17.595405 27583 raft_consensus.cc:2454] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.595554 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet abaaf307b73e46b6a5548a0acba68ba9
I20240503 02:09:17.595572 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c with delete_type TABLET_DATA_DELETED (replaced by tablet 2834e40454ef48429c31b50da5d2ab0f) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.595621 27197 catalog_manager.cc:6554] ReplaceTablet: tablet abaaf307b73e46b6a5548a0acba68ba9 deleted and replaced by tablet 7078864d79114bbe910112efb8ca01c3
I20240503 02:09:17.595638 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet a3650180caf546878779d1c89950b2c1 with delete_type TABLET_DATA_DELETED (replaced by tablet 31b262a1decf457ebab85ea106273fe2) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.595687 28661 tablet_replica.cc:330] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.595731 28661 raft_consensus.cc:2229] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.595777 28661 raft_consensus.cc:2258] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.595867 28661 ts_tablet_manager.cc:1875] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.596025 29240 consensus_queue.cc:1035] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.596328 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c with delete_type TABLET_DATA_DELETED (replaced by tablet 2834e40454ef48429c31b50da5d2ab0f) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.596397 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7078864d79114bbe910112efb8ca01c3) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.596439 28653 tablet_replica.cc:330] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.596460 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet a3650180caf546878779d1c89950b2c1 with delete_type TABLET_DATA_DELETED (replaced by tablet 31b262a1decf457ebab85ea106273fe2) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.596483 28653 raft_consensus.cc:2229] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.596520 28653 raft_consensus.cc:2258] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.596612 28653 ts_tablet_manager.cc:1875] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.597018 28664 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.597057 28664 ts_tablet_manager.cc:1367] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.000s
I20240503 02:09:17.597134 28661 ts_tablet_manager.cc:1888] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.597178 28661 log.cc:1196] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8a5b3f2a01b5453983b0d767ff376a6c
I20240503 02:09:17.597198 28664 raft_consensus.cc:348] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.597250 28664 raft_consensus.cc:374] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.597266 28664 raft_consensus.cc:729] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.597301 28664 consensus_queue.cc:260] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.597378 28661 ts_tablet_manager.cc:1909] T 8a5b3f2a01b5453983b0d767ff376a6c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.597416 28664 ts_tablet_manager.cc:1398] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.597472 28664 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.597517 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7078864d79114bbe910112efb8ca01c3) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.597806 28664 tablet_bootstrap.cc:654] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.595150 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "389839158b314fd7a34852af7b4afadc" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.597923 28661 tablet_replica.cc:330] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.597913 27587 raft_consensus.cc:3037] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.597936 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet a3650180caf546878779d1c89950b2c1 with delete_type TABLET_DATA_DELETED (replaced by tablet 31b262a1decf457ebab85ea106273fe2) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.597970 28661 raft_consensus.cc:2229] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.598004 28654 tablet_replica.cc:330] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.598024 28661 raft_consensus.cc:2258] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.598057 28654 raft_consensus.cc:2229] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.598084 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet abaaf307b73e46b6a5548a0acba68ba9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7078864d79114bbe910112efb8ca01c3) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.598107 28654 raft_consensus.cc:2258] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.598269 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.598297 28661 ts_tablet_manager.cc:1875] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.598555 27587 raft_consensus.cc:2454] T 389839158b314fd7a34852af7b4afadc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.597923 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5b3f2a01b5453983b0d767ff376a6c with delete_type TABLET_DATA_DELETED (replaced by tablet 2834e40454ef48429c31b50da5d2ab0f) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.598749 28507 raft_consensus.cc:482] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.598783 28507 raft_consensus.cc:504] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.598839 28655 tablet_bootstrap.cc:492] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.598870 28507 leader_election.cc:288] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.598875 28655 ts_tablet_manager.cc:1367] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.001s
I20240503 02:09:17.598995 28655 raft_consensus.cc:348] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.599037 28655 raft_consensus.cc:374] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.599054 28655 raft_consensus.cc:729] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.599078 28655 consensus_queue.cc:260] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.599159 28655 ts_tablet_manager.cc:1398] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.599212 28655 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.599465 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2acfdf916c40878a51a85eaadf7dbf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.599545 28655 tablet_bootstrap.cc:654] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.599560 28653 ts_tablet_manager.cc:1888] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.599598 28653 log.cc:1196] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8a5b3f2a01b5453983b0d767ff376a6c
I20240503 02:09:17.599640 28654 ts_tablet_manager.cc:1875] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.599771 28653 ts_tablet_manager.cc:1909] T 8a5b3f2a01b5453983b0d767ff376a6c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.599968 28653 tablet_replica.cc:330] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.600028 28653 raft_consensus.cc:2229] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
W20240503 02:09:17.600020 27523 leader_election.cc:341] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.600065 28653 pending_rounds.cc:62] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:17.600085 28653 pending_rounds.cc:68] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420036738662400 op_type: NO_OP noop_request { }
I20240503 02:09:17.600124 28653 raft_consensus.cc:2866] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.600144 28653 raft_consensus.cc:2258] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.600818 28654 ts_tablet_manager.cc:1888] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.600867 28654 log.cc:1196] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a3650180caf546878779d1c89950b2c1
I20240503 02:09:17.601042 28654 ts_tablet_manager.cc:1909] T a3650180caf546878779d1c89950b2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.601253 28653 ts_tablet_manager.cc:1875] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.601527 28664 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.601564 28664 ts_tablet_manager.cc:1367] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:17.601686 28664 raft_consensus.cc:348] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.601732 28664 raft_consensus.cc:374] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.601747 28664 raft_consensus.cc:729] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.601779 28664 consensus_queue.cc:260] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.601855 28664 ts_tablet_manager.cc:1398] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.601903 28664 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.602210 28664 tablet_bootstrap.cc:654] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.602291 28653 ts_tablet_manager.cc:1888] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.602324 28653 log.cc:1196] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/abaaf307b73e46b6a5548a0acba68ba9
I20240503 02:09:17.602448 28653 ts_tablet_manager.cc:1909] T abaaf307b73e46b6a5548a0acba68ba9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.602780 28654 tablet_replica.cc:330] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.602842 28654 raft_consensus.cc:2229] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.602870 28654 raft_consensus.cc:2258] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.602911 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2acfdf916c40878a51a85eaadf7dbf" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.603091 28653 tablet_replica.cc:330] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.603152 28653 raft_consensus.cc:2229] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.603183 28653 raft_consensus.cc:2258] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.603277 28654 ts_tablet_manager.cc:1875] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.603334 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:17.603401 28653 ts_tablet_manager.cc:1875] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.604396 28654 ts_tablet_manager.cc:1888] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.604449 28654 log.cc:1196] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/abaaf307b73e46b6a5548a0acba68ba9
I20240503 02:09:17.604532 28653 ts_tablet_manager.cc:1888] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.604566 28653 log.cc:1196] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a3650180caf546878779d1c89950b2c1
I20240503 02:09:17.604594 28654 ts_tablet_manager.cc:1909] T abaaf307b73e46b6a5548a0acba68ba9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.604696 28653 ts_tablet_manager.cc:1909] T a3650180caf546878779d1c89950b2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.604946 29241 consensus_queue.cc:1035] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.605057 28656 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.605094 28656 ts_tablet_manager.cc:1367] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240503 02:09:17.605209 28656 raft_consensus.cc:348] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.605255 28656 raft_consensus.cc:374] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.605271 28656 raft_consensus.cc:729] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.605295 28656 consensus_queue.cc:260] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.605420 28656 ts_tablet_manager.cc:1398] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.605559 29558 tablet.cc:1722] T 6dcd0014c93f4a3296e33c7d3221e06e P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.606426 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet abaaf307b73e46b6a5548a0acba68ba9 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.606503 28654 tablet_replica.cc:330] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.606554 28654 raft_consensus.cc:2229] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.606585 28654 raft_consensus.cc:2258] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.606705 28654 ts_tablet_manager.cc:1875] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.606753 30009 tablet.cc:1722] T 6dcd0014c93f4a3296e33c7d3221e06e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20240503 02:09:17.607300 27524 leader_election.cc:341] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.607357 27524 leader_election.cc:302] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.607647 28507 raft_consensus.cc:2726] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.607892 28654 ts_tablet_manager.cc:1888] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.607932 28654 log.cc:1196] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8a5b3f2a01b5453983b0d767ff376a6c
I20240503 02:09:17.608062 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.608100 28654 ts_tablet_manager.cc:1909] T 8a5b3f2a01b5453983b0d767ff376a6c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.608117 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet abaaf307b73e46b6a5548a0acba68ba9 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.608157 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:17.608465 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8a5b3f2a01b5453983b0d767ff376a6c (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.609704 28655 tablet_bootstrap.cc:492] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.609745 28655 ts_tablet_manager.cc:1367] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:17.609825 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.609879 28655 raft_consensus.cc:348] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.609926 28655 raft_consensus.cc:374] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.609942 28655 raft_consensus.cc:729] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.609974 28655 consensus_queue.cc:260] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.610052 28655 ts_tablet_manager.cc:1398] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.610105 28655 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.610057 27195 catalog_manager.cc:5484] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.599488 28661 ts_tablet_manager.cc:1888] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.610188 28661 log.cc:1196] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a3650180caf546878779d1c89950b2c1
I20240503 02:09:17.610445 28655 tablet_bootstrap.cc:654] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.610416 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 7078864d79114bbe910112efb8ca01c3 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.610441 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 2834e40454ef48429c31b50da5d2ab0f (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.610435 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 31b262a1decf457ebab85ea106273fe2 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.610636 28661 ts_tablet_manager.cc:1909] T a3650180caf546878779d1c89950b2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.610816 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 31b262a1decf457ebab85ea106273fe2 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.610924 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b262a1decf457ebab85ea106273fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.611178 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a3650180caf546878779d1c89950b2c1 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]) successfully deleted
I20240503 02:09:17.611258 28661 tablet_replica.cc:330] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.611351 28661 raft_consensus.cc:2229] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.611407 28661 raft_consensus.cc:2258] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.611673 28661 ts_tablet_manager.cc:1875] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.611711 28664 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.611743 28664 ts_tablet_manager.cc:1367] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240503 02:09:17.611871 28664 raft_consensus.cc:348] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.611937 28664 raft_consensus.cc:374] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.611954 28664 raft_consensus.cc:729] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.611989 28664 consensus_queue.cc:260] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.612017 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 7078864d79114bbe910112efb8ca01c3 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.612084 28664 ts_tablet_manager.cc:1398] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.612161 28664 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.612180 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 2834e40454ef48429c31b50da5d2ab0f (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.612093 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7078864d79114bbe910112efb8ca01c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.612427 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2834e40454ef48429c31b50da5d2ab0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.612633 28664 tablet_bootstrap.cc:654] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.610517 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7078864d79114bbe910112efb8ca01c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.614089 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2834e40454ef48429c31b50da5d2ab0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.614953 28656 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.615000 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b262a1decf457ebab85ea106273fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.615427 28656 tablet_bootstrap.cc:654] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.615934 28656 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.615968 28656 ts_tablet_manager.cc:1367] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.616084 28656 raft_consensus.cc:348] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.616151 28656 raft_consensus.cc:374] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.616170 28656 raft_consensus.cc:729] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.616196 28656 consensus_queue.cc:260] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.616277 28656 ts_tablet_manager.cc:1398] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.616346 28656 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.616657 28656 tablet_bootstrap.cc:654] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.618039 28661 ts_tablet_manager.cc:1888] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.618866 28661 log.cc:1196] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/abaaf307b73e46b6a5548a0acba68ba9
I20240503 02:09:17.618961 28710 tablet.cc:1722] T 6dcd0014c93f4a3296e33c7d3221e06e P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.619650 28661 ts_tablet_manager.cc:1909] T abaaf307b73e46b6a5548a0acba68ba9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.619972 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet abaaf307b73e46b6a5548a0acba68ba9 (table default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]) successfully deleted
I20240503 02:09:17.620501 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 31b262a1decf457ebab85ea106273fe2 (DEFAULT_TABLE table=default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.620553 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7078864d79114bbe910112efb8ca01c3 (DEFAULT_TABLE table=default.Table_9f8a89570cd247eabbcde7bb9838a07e [id=65ce5fbf71574308973d763cc1e9748d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.620599 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b262a1decf457ebab85ea106273fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.620649 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7078864d79114bbe910112efb8ca01c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.621037 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_b72419469762486eb080ac91f57505df [id=e2143b8ffeae471d93521e4896bb63a3] alter complete (version 1)
W20240503 02:09:17.621090 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 4d471aedbe2a45369909f9590e3dfb83 consensus state has no leader
I20240503 02:09:17.621937 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 2834e40454ef48429c31b50da5d2ab0f (DEFAULT_TABLE table=default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.622032 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2834e40454ef48429c31b50da5d2ab0f. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:17.622586 27195 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:17.623466 28664 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.623512 28664 ts_tablet_manager.cc:1367] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:17.623632 28664 raft_consensus.cc:348] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.623700 28664 raft_consensus.cc:374] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.623718 28664 raft_consensus.cc:729] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.623749 28664 consensus_queue.cc:260] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.623838 28664 ts_tablet_manager.cc:1398] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.623899 28664 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.624315 28664 tablet_bootstrap.cc:654] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.624784 28664 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.624819 28664 ts_tablet_manager.cc:1367] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:17.624954 28664 raft_consensus.cc:348] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.625007 28664 raft_consensus.cc:374] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.625041 28664 raft_consensus.cc:729] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.625078 28664 consensus_queue.cc:260] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.625154 28664 ts_tablet_manager.cc:1398] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.625676 28664 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.626029 28664 tablet_bootstrap.cc:654] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.627036 28664 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.627050 28656 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.627071 28664 ts_tablet_manager.cc:1367] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.627084 28656 ts_tablet_manager.cc:1367] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:17.627189 28664 raft_consensus.cc:348] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.627204 28656 raft_consensus.cc:348] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.627236 28664 raft_consensus.cc:374] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.627254 28664 raft_consensus.cc:729] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.627254 28656 raft_consensus.cc:374] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.627271 28656 raft_consensus.cc:729] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.627303 28656 consensus_queue.cc:260] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.627291 28664 consensus_queue.cc:260] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.627386 28656 ts_tablet_manager.cc:1398] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.627385 28664 ts_tablet_manager.cc:1398] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.627439 28664 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.627454 28656 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.627800 28664 tablet_bootstrap.cc:654] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.627800 28656 tablet_bootstrap.cc:654] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.629747 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_597335db18b74d859f4f7a0fcd41def7" } modify_external_catalogs: true
I20240503 02:09:17.629817 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_597335db18b74d859f4f7a0fcd41def7" } modify_external_catalogs: true
I20240503 02:09:17.629930 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.630144 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.630224 28664 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.630256 28664 ts_tablet_manager.cc:1367] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:17.630381 28664 raft_consensus.cc:348] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.630445 28664 raft_consensus.cc:374] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.630468 28664 raft_consensus.cc:729] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.630501 28664 consensus_queue.cc:260] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.630605 28664 ts_tablet_manager.cc:1398] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.630918 28664 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.631345 28664 tablet_bootstrap.cc:654] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.631896 28664 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.631934 28664 ts_tablet_manager.cc:1367] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.631947 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
W20240503 02:09:17.632021 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 31b262a1decf457ebab85ea106273fe2 consensus state has no leader
I20240503 02:09:17.632043 28664 raft_consensus.cc:348] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.632097 28664 raft_consensus.cc:374] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.632113 28664 raft_consensus.cc:729] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.632144 28664 consensus_queue.cc:260] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.632228 28664 ts_tablet_manager.cc:1398] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.632277 28664 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.632596 28664 tablet_bootstrap.cc:654] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.632642 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
W20240503 02:09:17.632714 27197 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 2834e40454ef48429c31b50da5d2ab0f consensus state has no leader
I20240503 02:09:17.632886 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:17.632957 28656 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.632994 28656 ts_tablet_manager.cc:1367] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:17.633343 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f54b533856464e81d77e726b2b7f31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.633435 28654 tablet_replica.cc:330] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.633435 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f54b533856464e81d77e726b2b7f31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.633496 28654 raft_consensus.cc:2229] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.633527 28654 raft_consensus.cc:2258] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.633628 28654 ts_tablet_manager.cc:1875] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.633881 28656 raft_consensus.cc:348] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.633949 28656 raft_consensus.cc:374] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.633966 28656 raft_consensus.cc:729] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.634001 28656 consensus_queue.cc:260] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.634078 28656 ts_tablet_manager.cc:1398] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.634855 28661 tablet_replica.cc:330] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.634908 28661 raft_consensus.cc:2229] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.634933 28661 raft_consensus.cc:2258] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.634974 28654 ts_tablet_manager.cc:1888] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.635017 28654 log.cc:1196] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:17.635047 28661 ts_tablet_manager.cc:1875] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.635205 28654 ts_tablet_manager.cc:1909] T f4f54b533856464e81d77e726b2b7f31 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.635618 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f4f54b533856464e81d77e726b2b7f31 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:17.636180 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ca2d781a10ed43a999c3abfa17fc4f91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.636293 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.636391 28661 ts_tablet_manager.cc:1888] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.636431 28661 log.cc:1196] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/37ed1b8a0ae9439a8a7f8824eb68e01e
I20240503 02:09:17.636564 28661 ts_tablet_manager.cc:1909] T 37ed1b8a0ae9439a8a7f8824eb68e01e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.636873 28664 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.636911 28664 ts_tablet_manager.cc:1367] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:17.637022 28664 raft_consensus.cc:348] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.637065 28664 raft_consensus.cc:374] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.637081 28664 raft_consensus.cc:729] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.637113 28664 consensus_queue.cc:260] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.637213 28664 ts_tablet_manager.cc:1398] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.637575 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet f4f54b533856464e81d77e726b2b7f31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.637655 28653 tablet_replica.cc:330] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.637712 28661 tablet_replica.cc:330] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.637718 28653 raft_consensus.cc:2229] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.637763 28661 raft_consensus.cc:2229] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.637784 28653 raft_consensus.cc:2258] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.637795 28661 raft_consensus.cc:2258] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.637889 28661 ts_tablet_manager.cc:1875] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.637881 28653 ts_tablet_manager.cc:1875] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.638311 27201 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.638433 28507 raft_consensus.cc:482] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.638473 28507 raft_consensus.cc:504] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.638579 28507 leader_election.cc:288] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.638751 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2834e40454ef48429c31b50da5d2ab0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.638810 27720 raft_consensus.cc:2454] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.638835 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.638870 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:17.639004 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (table default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]) successfully deleted
I20240503 02:09:17.639168 27523 leader_election.cc:302] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.639238 28507 raft_consensus.cc:2781] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.639261 28507 raft_consensus.cc:482] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.639279 28507 raft_consensus.cc:3037] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.639328 28653 ts_tablet_manager.cc:1888] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.639364 28653 log.cc:1196] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:17.639521 28653 ts_tablet_manager.cc:1909] T f4f54b533856464e81d77e726b2b7f31 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.639729 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2834e40454ef48429c31b50da5d2ab0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.639896 28507 raft_consensus.cc:504] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.640041 28507 leader_election.cc:288] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:17.640202 27524 leader_election.cc:341] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.640192 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2834e40454ef48429c31b50da5d2ab0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.640362 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2834e40454ef48429c31b50da5d2ab0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.640416 27720 raft_consensus.cc:3037] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.640466 28409 consensus_queue.cc:1035] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.640559 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:17.640918 28661 ts_tablet_manager.cc:1888] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20240503 02:09:17.640935 27524 leader_election.cc:341] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.640957 28661 log.cc:1196] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f4f54b533856464e81d77e726b2b7f31
I20240503 02:09:17.641006 27720 raft_consensus.cc:2454] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.641126 28661 ts_tablet_manager.cc:1909] T f4f54b533856464e81d77e726b2b7f31 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.641542 27523 leader_election.cc:302] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.641677 28410 raft_consensus.cc:2781] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.641723 28410 raft_consensus.cc:686] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.641762 28410 consensus_queue.cc:237] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.642110 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f4f54b533856464e81d77e726b2b7f31 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
W20240503 02:09:17.643229 27190 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4d471aedbe2a45369909f9590e3dfb83 consensus state has no leader
I20240503 02:09:17.643564 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f4f54b533856464e81d77e726b2b7f31 (table default.Table_597335db18b74d859f4f7a0fcd41def7 [id=323340acba774170949bdf6bf1007eb0]) successfully deleted
I20240503 02:09:17.646353 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.646409 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.646461 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48620839bd8d4bbd9f655c1b8aa180af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.646495 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48620839bd8d4bbd9f655c1b8aa180af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.647604 28664 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.647882 27195 catalog_manager.cc:5484] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.647944 28664 tablet_bootstrap.cc:654] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.647982 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:17.648195 28656 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.648550 28656 tablet_bootstrap.cc:654] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.648984 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.649096 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48620839bd8d4bbd9f655c1b8aa180af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.650914 28409 consensus_queue.cc:1035] T 4a1a5c30890d491f979cdb870dbdcf76 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.651415 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9017bb9c723a401c9008d8ef638838ea" } modify_external_catalogs: true
I20240503 02:09:17.651473 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9017bb9c723a401c9008d8ef638838ea" } modify_external_catalogs: true
I20240503 02:09:17.653594 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e776dc8757154957900b9fdecbe4dcc1
I20240503 02:09:17.653832 28664 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.653877 28664 ts_tablet_manager.cc:1367] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.653999 28664 raft_consensus.cc:348] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.654060 28664 raft_consensus.cc:374] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.654024 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet e776dc8757154957900b9fdecbe4dcc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.654083 28664 raft_consensus.cc:729] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.654115 28661 tablet_replica.cc:330] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.654127 28664 consensus_queue.cc:260] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.654170 28661 raft_consensus.cc:2229] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.654201 28664 ts_tablet_manager.cc:1398] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.654219 28661 raft_consensus.cc:2258] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.654383 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet e776dc8757154957900b9fdecbe4dcc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.654470 28654 tablet_replica.cc:330] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.654523 28654 raft_consensus.cc:2229] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.654551 28654 raft_consensus.cc:2258] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.654678 28661 ts_tablet_manager.cc:1875] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.654754 28654 ts_tablet_manager.cc:1875] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.656033 28661 ts_tablet_manager.cc:1888] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.656044 28654 ts_tablet_manager.cc:1888] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.656073 28661 log.cc:1196] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/e776dc8757154957900b9fdecbe4dcc1
I20240503 02:09:17.656086 28654 log.cc:1196] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/e776dc8757154957900b9fdecbe4dcc1
I20240503 02:09:17.656250 28661 ts_tablet_manager.cc:1909] T e776dc8757154957900b9fdecbe4dcc1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.656289 28654 ts_tablet_manager.cc:1909] T e776dc8757154957900b9fdecbe4dcc1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.656554 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet e776dc8757154957900b9fdecbe4dcc1 (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:17.657207 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet e776dc8757154957900b9fdecbe4dcc1 (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:17.657346 28656 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.657389 28656 ts_tablet_manager.cc:1367] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:17.657500 28656 raft_consensus.cc:348] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.657543 28656 raft_consensus.cc:374] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.657559 28656 raft_consensus.cc:729] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.657593 28656 consensus_queue.cc:260] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.657668 28656 ts_tablet_manager.cc:1398] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.658119 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b737687622d24ad89da74d6418ce396e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.658224 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b737687622d24ad89da74d6418ce396e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.659235 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet e776dc8757154957900b9fdecbe4dcc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.659315 28653 tablet_replica.cc:330] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.659385 28653 raft_consensus.cc:2229] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.659413 28653 raft_consensus.cc:2258] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.659579 28653 ts_tablet_manager.cc:1875] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.660811 28653 ts_tablet_manager.cc:1888] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.660868 28653 log.cc:1196] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/e776dc8757154957900b9fdecbe4dcc1
I20240503 02:09:17.661056 28653 ts_tablet_manager.cc:1909] T e776dc8757154957900b9fdecbe4dcc1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.661476 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet e776dc8757154957900b9fdecbe4dcc1 (table default.Table_9017bb9c723a401c9008d8ef638838ea [id=6188dd39158545d490b7352e0c58d2a3]) successfully deleted
I20240503 02:09:17.662112 27702 tablet_service.cc:1467] Processing CreateTablet for tablet efefb11948624bdfb610a14f81c8c976 (DEFAULT_TABLE table=default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.662163 27560 tablet_service.cc:1467] Processing CreateTablet for tablet efefb11948624bdfb610a14f81c8c976 (DEFAULT_TABLE table=default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.662212 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efefb11948624bdfb610a14f81c8c976. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.662249 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efefb11948624bdfb610a14f81c8c976. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.663218 28664 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.663537 28664 tablet_bootstrap.cc:654] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.663792 27426 tablet_service.cc:1467] Processing CreateTablet for tablet efefb11948624bdfb610a14f81c8c976 (DEFAULT_TABLE table=default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.663877 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efefb11948624bdfb610a14f81c8c976. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.664999 28656 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.665369 28656 tablet_bootstrap.cc:654] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.667899 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0210f27a4c6348b9bd2efc147d1fc688"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.667999 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0210f27a4c6348b9bd2efc147d1fc688 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.668352 28664 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.668394 28664 ts_tablet_manager.cc:1367] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:17.668587 28664 raft_consensus.cc:348] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.668641 28664 raft_consensus.cc:374] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.668659 28664 raft_consensus.cc:729] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.668706 28664 consensus_queue.cc:260] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.668808 28664 ts_tablet_manager.cc:1398] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.670922 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e12d823f5eb49f38d5fdc7ee2185bb4. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:17.671003 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P c324c9220fd14d3c817b18bbd3774439 [sys.catalog]: This master's current role is: FOLLOWER
I20240503 02:09:17.671427 28655 tablet_bootstrap.cc:492] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.671473 28655 ts_tablet_manager.cc:1367] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.061s user 0.000s sys 0.001s
I20240503 02:09:17.671490 28617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c324c9220fd14d3c817b18bbd3774439" member_type: VOTER last_known_addr { host: "127.26.117.62" port: 37529 } } peers { permanent_uuid: "9e12d823f5eb49f38d5fdc7ee2185bb4" member_type: VOTER last_known_addr { host: "127.26.117.61" port: 37847 } } peers { permanent_uuid: "9c75a5f0d0824fafbc14d2a2c1613db1" member_type: VOTER last_known_addr { host: "127.26.117.60" port: 36369 } } }
I20240503 02:09:17.671561 28617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4 [sys.catalog]: This master's current role is: LEADER
I20240503 02:09:17.671567 28655 raft_consensus.cc:348] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.671622 28655 raft_consensus.cc:374] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.671638 28655 raft_consensus.cc:729] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.671674 28655 consensus_queue.cc:260] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.671765 28655 ts_tablet_manager.cc:1398] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.671787 28656 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.671830 28655 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.671841 28656 ts_tablet_manager.cc:1367] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:17.671965 28656 raft_consensus.cc:348] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.672012 28656 raft_consensus.cc:374] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.672029 28656 raft_consensus.cc:729] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.672050 28655 tablet_bootstrap.cc:654] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.672068 28656 consensus_queue.cc:260] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.672154 28656 ts_tablet_manager.cc:1398] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.672412 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d35c89a1c8454483aacc3d4449a5afed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.672508 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d35c89a1c8454483aacc3d4449a5afed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.673193 28409 consensus_queue.cc:1035] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.675050 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_4ff567fc40464d588787c5b5390948ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.675144 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ff567fc40464d588787c5b5390948ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.677886 27560 tablet_service.cc:1467] Processing CreateTablet for tablet dba5e73b037d400ca2417c21f81729a6 (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.678192 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba5e73b037d400ca2417c21f81729a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.679442 28656 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.680105 27702 tablet_service.cc:1467] Processing CreateTablet for tablet dba5e73b037d400ca2417c21f81729a6 (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.680402 28507 raft_consensus.cc:482] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.680491 27426 tablet_service.cc:1467] Processing CreateTablet for tablet dba5e73b037d400ca2417c21f81729a6 (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.680737 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba5e73b037d400ca2417c21f81729a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.680552 28507 raft_consensus.cc:504] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.680684 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba5e73b037d400ca2417c21f81729a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.681866 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f36f9ab6b4fa28f6fb02a48bfd2ab" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.681927 27449 raft_consensus.cc:2454] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.681183 28766 raft_consensus.cc:482] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.682528 28766 raft_consensus.cc:504] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.682572 28655 tablet_bootstrap.cc:492] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.682618 28766 leader_election.cc:288] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.682673 28655 ts_tablet_manager.cc:1367] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:17.682897 28655 raft_consensus.cc:348] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.682966 28655 raft_consensus.cc:374] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.682987 28655 raft_consensus.cc:729] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.683022 28655 consensus_queue.cc:260] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.683236 28655 ts_tablet_manager.cc:1398] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.683293 28655 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.683533 28655 tablet_bootstrap.cc:654] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.684715 28656 tablet_bootstrap.cc:654] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.684278 28507 leader_election.cc:288] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.685343 28656 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.685384 28656 ts_tablet_manager.cc:1367] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:17.685422 27524 leader_election.cc:302] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
W20240503 02:09:17.685483 27524 consensus_peers.cc:485] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Couldn't send request to peer bd4b79bd8b57443faa6fa5e464570188. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:17.685520 28656 raft_consensus.cc:348] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.685568 28656 raft_consensus.cc:374] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.685556 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a7970b0e147358dab5f6814502ef0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.685586 28656 raft_consensus.cc:729] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.685598 27583 raft_consensus.cc:2454] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.685619 28656 consensus_queue.cc:260] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.685715 29240 raft_consensus.cc:2781] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.685732 28656 ts_tablet_manager.cc:1398] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.686749 28664 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.687086 28664 tablet_bootstrap.cc:654] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.685741 29240 raft_consensus.cc:482] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.687605 29240 raft_consensus.cc:3037] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.688074 29240 raft_consensus.cc:504] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.688140 29240 leader_election.cc:288] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.688596 28766 consensus_queue.cc:1035] T a7116932a247407196f5c17ddc1b4e9f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.688917 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9957972e38de45fb97dc8e24a14d8314"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.689010 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9957972e38de45fb97dc8e24a14d8314 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.689131 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a7970b0e147358dab5f6814502ef0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.689186 27720 raft_consensus.cc:2454] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.689749 27388 leader_election.cc:302] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.689725 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_7c554f7ea6f74725a303fabd9ae42cda"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.689834 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c554f7ea6f74725a303fabd9ae42cda in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.689837 28767 raft_consensus.cc:2781] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.689867 28767 raft_consensus.cc:482] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.689883 28767 raft_consensus.cc:3037] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.690299 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 36072c70fb464c9786bf4fef8f285acb (DEFAULT_TABLE table=default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.690415 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36072c70fb464c9786bf4fef8f285acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.690474 28767 raft_consensus.cc:504] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.690568 28767 leader_election.cc:288] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.690739 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 36072c70fb464c9786bf4fef8f285acb (DEFAULT_TABLE table=default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.690824 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36072c70fb464c9786bf4fef8f285acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.691643 28656 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.691843 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f36f9ab6b4fa28f6fb02a48bfd2ab" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.691902 27449 raft_consensus.cc:3037] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.692000 28656 tablet_bootstrap.cc:654] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.692158 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a7970b0e147358dab5f6814502ef0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.692234 27720 raft_consensus.cc:3037] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.692353 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51a7970b0e147358dab5f6814502ef0" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.692420 27587 raft_consensus.cc:3037] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.693022 27587 raft_consensus.cc:2454] T e51a7970b0e147358dab5f6814502ef0 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.693173 27449 raft_consensus.cc:2454] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.693725 27720 raft_consensus.cc:2454] T e51a7970b0e147358dab5f6814502ef0 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.693977 27388 leader_election.cc:302] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.693961 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ed1b8a0ae9439a8a7f8824eb68e01e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.694058 28766 raft_consensus.cc:2781] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.694090 28766 raft_consensus.cc:686] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.694094 28653 tablet_replica.cc:330] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.694134 28766 consensus_queue.cc:237] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.694160 28653 raft_consensus.cc:2229] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.694193 28653 raft_consensus.cc:2258] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.694324 28653 ts_tablet_manager.cc:1875] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.695042 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_def0b15b29374976bcd2300bc83f9c81" } new_table_name: "default.Table_42d2f6e69a48457b9427d3d00bd645eb" modify_external_catalogs: true
I20240503 02:09:17.695288 28656 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.695324 28656 ts_tablet_manager.cc:1367] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.695442 28656 raft_consensus.cc:348] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.695511 28656 raft_consensus.cc:374] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.695530 28656 raft_consensus.cc:729] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.695544 28653 ts_tablet_manager.cc:1888] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.695556 28656 consensus_queue.cc:260] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.695603 28653 log.cc:1196] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/37ed1b8a0ae9439a8a7f8824eb68e01e
I20240503 02:09:17.695654 28656 ts_tablet_manager.cc:1398] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.695753 28653 ts_tablet_manager.cc:1909] T 37ed1b8a0ae9439a8a7f8824eb68e01e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.696439 28664 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.696480 28664 ts_tablet_manager.cc:1367] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:17.696621 28664 raft_consensus.cc:348] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.696676 28664 raft_consensus.cc:374] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.696693 28664 raft_consensus.cc:729] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.696725 28664 consensus_queue.cc:260] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.696839 28664 ts_tablet_manager.cc:1398] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.696895 28664 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.697240 28664 tablet_bootstrap.cc:654] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.697953 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 37ed1b8a0ae9439a8a7f8824eb68e01e (table default.Table_83e83f7c5d1049358d61e4f86a74c868 [id=e12753992d46440e9b392ccbf65c6444]) successfully deleted
I20240503 02:09:17.698037 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (DEFAULT_TABLE table=default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.698122 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.698837 27524 leader_election.cc:302] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.698958 28507 raft_consensus.cc:2781] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.698997 28507 raft_consensus.cc:686] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.699028 28507 consensus_queue.cc:237] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.699680 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 7a23e797aee94bcb90cbb7545147a893 (DEFAULT_TABLE table=default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.699780 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a23e797aee94bcb90cbb7545147a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.699941 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 7a23e797aee94bcb90cbb7545147a893 (DEFAULT_TABLE table=default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.700028 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a23e797aee94bcb90cbb7545147a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.700501 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7a559ab04d7c4b148c054656cc3863db (DEFAULT_TABLE table=default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.700582 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a559ab04d7c4b148c054656cc3863db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.700899 27853 raft_consensus.cc:482] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.700942 27853 raft_consensus.cc:504] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.700987 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 7a559ab04d7c4b148c054656cc3863db (DEFAULT_TABLE table=default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.701026 27853 leader_election.cc:288] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.701069 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a559ab04d7c4b148c054656cc3863db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.701297 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b148a22d0440457daff31b9cf7d84da0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:17.701368 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5707b69d91141ddbc3e9e8d8c3af477" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.701418 27451 raft_consensus.cc:2454] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
W20240503 02:09:17.701419 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b148a22d0440457daff31b9cf7d84da0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.701642 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_cf18641e208546c391738b4e80b51bd7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:17.701690 28656 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
W20240503 02:09:17.701722 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf18641e208546c391738b4e80b51bd7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.702023 28656 tablet_bootstrap.cc:654] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.702092 27197 catalog_manager.cc:5484] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.702258 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 7a559ab04d7c4b148c054656cc3863db (DEFAULT_TABLE table=default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.702297 28710 tablet.cc:1722] T 12d29bd82e234761a76b1fc59cedbe1c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.702358 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a559ab04d7c4b148c054656cc3863db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.703356 29558 tablet.cc:1722] T 12d29bd82e234761a76b1fc59cedbe1c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.705595 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 7a23e797aee94bcb90cbb7545147a893 (DEFAULT_TABLE table=default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.705730 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (DEFAULT_TABLE table=default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.705811 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.705935 29551 consensus_queue.cc:1035] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.706055 27659 leader_election.cc:302] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.706116 27853 raft_consensus.cc:2781] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.706136 27853 raft_consensus.cc:482] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.706153 27853 raft_consensus.cc:3037] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.706141 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5707b69d91141ddbc3e9e8d8c3af477" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.706208 27587 raft_consensus.cc:2454] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.706729 27853 raft_consensus.cc:504] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.706822 28693 raft_consensus.cc:482] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.706861 28693 raft_consensus.cc:504] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.706964 28693 leader_election.cc:288] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.707598 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f36f9ab6b4fa28f6fb02a48bfd2ab" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.707656 27720 raft_consensus.cc:2454] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.707742 27720 raft_consensus.cc:1261] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.707784 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f36f9ab6b4fa28f6fb02a48bfd2ab" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.707839 27719 raft_consensus.cc:3037] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.708335 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (DEFAULT_TABLE table=default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.708396 27719 raft_consensus.cc:2454] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.708437 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.708475 27718 raft_consensus.cc:1261] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.709168 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99265e78c1d74facb95bb8f7abf71110" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.709225 27583 raft_consensus.cc:2454] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.709232 27562 tablet_service.cc:1467] Processing CreateTablet for tablet fada3a7eedae4a1c9f2372ea5699020f (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.709378 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fada3a7eedae4a1c9f2372ea5699020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.709975 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5707b69d91141ddbc3e9e8d8c3af477" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.710031 27587 raft_consensus.cc:3037] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.710398 27201 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_42d2f6e69a48457b9427d3d00bd645eb [id=af530775a302468796ce4cde6e07f9c9] alter complete (version 1)
I20240503 02:09:17.710630 27587 raft_consensus.cc:2454] T c5707b69d91141ddbc3e9e8d8c3af477 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.710716 27702 tablet_service.cc:1467] Processing CreateTablet for tablet fada3a7eedae4a1c9f2372ea5699020f (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.710764 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.710801 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fada3a7eedae4a1c9f2372ea5699020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.710850 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427f0063e66e4688be5c455dfdfe6e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.711329 27433 tablet_service.cc:1467] Processing CreateTablet for tablet fada3a7eedae4a1c9f2372ea5699020f (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.711385 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.711952 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.712045 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427f0063e66e4688be5c455dfdfe6e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.712260 28507 consensus_queue.cc:1035] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.712388 28507 consensus_queue.cc:1035] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.706832 27853 leader_election.cc:288] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.712836 30009 tablet.cc:1722] T 12d29bd82e234761a76b1fc59cedbe1c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.712962 27656 leader_election.cc:302] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.713037 27656 leader_election.cc:302] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.713086 28740 raft_consensus.cc:482] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.713111 28774 raft_consensus.cc:2781] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.713137 28774 raft_consensus.cc:482] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.713155 28774 raft_consensus.cc:3037] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.713127 28740 raft_consensus.cc:504] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.713236 28740 leader_election.cc:288] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.713269 28740 raft_consensus.cc:2781] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.713299 28740 raft_consensus.cc:686] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.713766 28774 raft_consensus.cc:504] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.713867 28774 leader_election.cc:288] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.714179 27450 raft_consensus.cc:1261] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.714239 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5707b69d91141ddbc3e9e8d8c3af477" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.714278 27448 raft_consensus.cc:3037] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.714267 28740 consensus_queue.cc:237] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.714843 27448 raft_consensus.cc:2454] T c5707b69d91141ddbc3e9e8d8c3af477 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.714912 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99265e78c1d74facb95bb8f7abf71110" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.714960 27447 raft_consensus.cc:2454] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.715407 27201 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5cfc506f31b1478d8d585da047adff86" } modify_external_catalogs: true
I20240503 02:09:17.715445 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_04b143bee9ff47b39766ba185154531c" } new_table_name: "default.Table_a604d0eec20e4037ad895d3802e5341f" modify_external_catalogs: true
I20240503 02:09:17.715462 27201 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5cfc506f31b1478d8d585da047adff86" } modify_external_catalogs: true
I20240503 02:09:17.716192 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 427f0063e66e4688be5c455dfdfe6e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.716780 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 53cd092cf71e47fcad1f2f9091953141
I20240503 02:09:17.717247 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a23e797aee94bcb90cbb7545147a893. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.717347 27433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fada3a7eedae4a1c9f2372ea5699020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.718083 28656 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.718127 28656 ts_tablet_manager.cc:1367] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:17.718246 28656 raft_consensus.cc:348] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.718297 28656 raft_consensus.cc:374] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.718313 28656 raft_consensus.cc:729] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.718330 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cd092cf71e47fcad1f2f9091953141 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.718349 28656 consensus_queue.cc:260] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.718427 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cd092cf71e47fcad1f2f9091953141 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.718446 28656 ts_tablet_manager.cc:1398] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.718511 28656 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.718621 28661 tablet_replica.cc:330] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.718677 28661 raft_consensus.cc:2229] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.718710 28661 raft_consensus.cc:2258] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.718863 28656 tablet_bootstrap.cc:654] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.718964 28661 ts_tablet_manager.cc:1875] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.719240 28654 tablet_replica.cc:330] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.719324 28654 raft_consensus.cc:2229] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.719341 28656 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.719367 28654 raft_consensus.cc:2258] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.719378 28656 ts_tablet_manager.cc:1367] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.719514 28656 raft_consensus.cc:348] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.719564 28656 raft_consensus.cc:374] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.719581 28656 raft_consensus.cc:729] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.719616 28656 consensus_queue.cc:260] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.719699 28656 ts_tablet_manager.cc:1398] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.719745 28656 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.719910 28654 ts_tablet_manager.cc:1875] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.720052 28656 tablet_bootstrap.cc:654] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.720208 28661 ts_tablet_manager.cc:1888] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.720252 28661 log.cc:1196] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/53cd092cf71e47fcad1f2f9091953141
I20240503 02:09:17.720425 28661 ts_tablet_manager.cc:1909] T 53cd092cf71e47fcad1f2f9091953141 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.721107 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:17.721263 28654 ts_tablet_manager.cc:1888] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.721307 28654 log.cc:1196] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/53cd092cf71e47fcad1f2f9091953141
I20240503 02:09:17.721545 28654 ts_tablet_manager.cc:1909] T 53cd092cf71e47fcad1f2f9091953141 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.722301 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:17.722553 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a874a3bdc66345b7b0d4c66ef9a9176c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.722656 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.722636 29643 consensus_queue.cc:1035] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.722719 29643 consensus_queue.cc:1035] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.722918 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99265e78c1d74facb95bb8f7abf71110" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.722968 27451 raft_consensus.cc:3037] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.723541 27451 raft_consensus.cc:2454] T 99265e78c1d74facb95bb8f7abf71110 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.723641 27449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba5e73b037d400ca2417c21f81729a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.723923 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 53cd092cf71e47fcad1f2f9091953141 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.723994 28653 tablet_replica.cc:330] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.724045 28653 raft_consensus.cc:2229] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.724104 28653 raft_consensus.cc:2258] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.724188 27201 catalog_manager.cc:5484] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.724261 28653 ts_tablet_manager.cc:1875] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.725560 28653 ts_tablet_manager.cc:1888] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.725615 28653 log.cc:1196] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/53cd092cf71e47fcad1f2f9091953141
I20240503 02:09:17.725792 28653 ts_tablet_manager.cc:1909] T 53cd092cf71e47fcad1f2f9091953141 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.726217 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.726302 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca02d87dfd549e9bf01c7f3d38928e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.726709 28410 consensus_queue.cc:1035] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.726799 28410 consensus_queue.cc:1035] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.726864 28410 consensus_queue.cc:1035] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.727349 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.727427 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca02d87dfd549e9bf01c7f3d38928e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.727726 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 36072c70fb464c9786bf4fef8f285acb (DEFAULT_TABLE table=default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.727805 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36072c70fb464c9786bf4fef8f285acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.729156 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99265e78c1d74facb95bb8f7abf71110" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.729152 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba5e73b037d400ca2417c21f81729a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.729276 27584 raft_consensus.cc:2454] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.729238 27583 raft_consensus.cc:3037] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.729203 29240 consensus_queue.cc:1035] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.729714 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 53cd092cf71e47fcad1f2f9091953141 (table default.Table_5cfc506f31b1478d8d585da047adff86 [id=2146a747ed554a8985a1650cba070530]) successfully deleted
I20240503 02:09:17.729856 27583 raft_consensus.cc:2454] T 99265e78c1d74facb95bb8f7abf71110 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.729892 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.729970 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca02d87dfd549e9bf01c7f3d38928e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.732069 27659 leader_election.cc:302] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
W20240503 02:09:17.732184 27659 leader_election.cc:341] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.732237 28774 raft_consensus.cc:2781] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.732272 28774 raft_consensus.cc:686] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.732311 28774 consensus_queue.cc:237] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.732825 29558 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.733373 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_956e18a0f62f41ad8a03bc02f603261b" } new_table_name: "default.Table_3bc8fca3155b4f5a982e9eb561b67c35" modify_external_catalogs: true
I20240503 02:09:17.733860 28693 raft_consensus.cc:482] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.733908 28693 raft_consensus.cc:504] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.734001 28693 leader_election.cc:288] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.734337 27848 consensus_queue.cc:1035] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.735291 27197 catalog_manager.cc:5484] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.736121 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5aa184df608d4c2dafb9752f9a6e615e" } new_table_name: "default.Table_d86e6ae97444426d955f743e6842a150" modify_external_catalogs: true
I20240503 02:09:17.736534 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9001e72388f7452e85476a5d2b3cf1c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.736604 27583 raft_consensus.cc:2454] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.736831 27656 leader_election.cc:302] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.736938 28693 raft_consensus.cc:2781] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.736968 28693 raft_consensus.cc:482] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.736984 28693 raft_consensus.cc:3037] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.737027 27448 raft_consensus.cc:1261] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.737614 28693 raft_consensus.cc:504] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.737726 28693 leader_election.cc:288] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.738106 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.738619 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9001e72388f7452e85476a5d2b3cf1c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.738670 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba5e73b037d400ca2417c21f81729a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.738684 27448 raft_consensus.cc:2454] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.739212 28710 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.741652 30009 tablet.cc:1722] T eb76d46144e74642bdcb0146ed419ce5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.741822 27717 raft_consensus.cc:1261] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.742486 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_a604d0eec20e4037ad895d3802e5341f [id=53e0fca749ea4dc699450e4503b21c03] alter complete (version 3)
I20240503 02:09:17.743628 28766 consensus_queue.cc:1035] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240503 02:09:17.744454 27659 leader_election.cc:341] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.744518 27659 leader_election.cc:302] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.744721 28774 raft_consensus.cc:2781] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.744761 28774 raft_consensus.cc:482] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.744779 28774 raft_consensus.cc:3037] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.745404 28774 raft_consensus.cc:504] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.745509 28774 leader_election.cc:288] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.745754 28507 consensus_queue.cc:1035] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.745972 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet eb76d46144e74642bdcb0146ed419ce5 (table default.Table_a43ecbfa64fb409b8c94d9c7adb9fbce [id=9a49cbefe8de4cb287512d93b353114d]). Expected version 1 got 0
I20240503 02:09:17.746019 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]). Expected version 1 got 0
I20240503 02:09:17.746049 27201 catalog_manager.cc:5484] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.746309 30009 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.747800 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9001e72388f7452e85476a5d2b3cf1c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.747872 27451 raft_consensus.cc:3037] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.748448 27451 raft_consensus.cc:2454] T 9001e72388f7452e85476a5d2b3cf1c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.748564 27659 leader_election.cc:302] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.748627 28774 raft_consensus.cc:2781] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.748659 28774 raft_consensus.cc:686] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.748693 28774 consensus_queue.cc:237] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.749100 29643 consensus_queue.cc:1035] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.749531 29558 tablet.cc:1722] T eb76d46144e74642bdcb0146ed419ce5 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.751566 28507 consensus_queue.cc:1035] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.751672 28507 consensus_queue.cc:1035] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.751736 28507 consensus_queue.cc:1035] T 43948f8039d948efbf1dcde10e012be4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.751796 28693 consensus_queue.cc:1035] T 0182ed8465eb40238a7efc2850f18173 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.751588 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba5e73b037d400ca2417c21f81729a6" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.751562 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9001e72388f7452e85476a5d2b3cf1c1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.752045 27583 raft_consensus.cc:3037] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.752709 27583 raft_consensus.cc:2454] T 9001e72388f7452e85476a5d2b3cf1c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.751974 27587 raft_consensus.cc:3037] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.753741 27587 raft_consensus.cc:2454] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.755779 30009 tablet.cc:1722] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:17.756774 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a2c00d21605f406d90c335256aa6cb1f" } new_table_name: "default.Table_9cb244f396f642349217ee1381639223" modify_external_catalogs: true
I20240503 02:09:17.757212 29240 consensus_queue.cc:1035] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.757302 29240 consensus_queue.cc:1035] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.757341 27656 leader_election.cc:302] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.757400 28774 consensus_queue.cc:1035] T 2983e9a39aed45f29aaaca9c5c010bb8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.757437 28693 consensus_queue.cc:1035] T cec4ddee00974610917b834da66647d3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.757476 28774 raft_consensus.cc:2781] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.757508 28774 raft_consensus.cc:686] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.757542 28774 consensus_queue.cc:237] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.758040 29558 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.758941 28710 tablet.cc:1722] T eb76d46144e74642bdcb0146ed419ce5 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.759793 27195 catalog_manager.cc:5484] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.759974 28664 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.760016 28664 ts_tablet_manager.cc:1367] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.063s user 0.001s sys 0.000s
I20240503 02:09:17.760149 28664 raft_consensus.cc:348] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.760198 28664 raft_consensus.cc:374] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.760214 28664 raft_consensus.cc:729] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.760248 28664 consensus_queue.cc:260] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.760349 28664 ts_tablet_manager.cc:1398] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.760352 28655 tablet_bootstrap.cc:492] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.760391 28655 ts_tablet_manager.cc:1367] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.077s user 0.000s sys 0.001s
I20240503 02:09:17.760402 28664 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.760485 28655 raft_consensus.cc:348] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.760540 28655 raft_consensus.cc:374] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.760557 28655 raft_consensus.cc:729] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.760596 28655 consensus_queue.cc:260] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.760676 28655 ts_tablet_manager.cc:1398] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.760722 28655 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.760747 28664 tablet_bootstrap.cc:654] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.760957 28655 tablet_bootstrap.cc:654] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.761178 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 48620839bd8d4bbd9f655c1b8aa180af from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.761305 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e7f335e7089a44c9aff8bc2ddf88f113" } modify_external_catalogs: true
I20240503 02:09:17.761376 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e7f335e7089a44c9aff8bc2ddf88f113" } modify_external_catalogs: true
I20240503 02:09:17.761976 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 58311afeda3d408f9a387c6ec2398e3c (table default.Table_3bc8fca3155b4f5a982e9eb561b67c35 [id=401e087f4afc46eda0d0f47b661c7f51]). Expected version 1 got 0
I20240503 02:09:17.763654 28656 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.763712 28656 ts_tablet_manager.cc:1367] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20240503 02:09:17.763840 28656 raft_consensus.cc:348] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.763896 28656 raft_consensus.cc:374] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.763913 28656 raft_consensus.cc:729] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.763948 28656 consensus_queue.cc:260] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.764043 28656 ts_tablet_manager.cc:1398] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.764086 28656 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.764137 27195 catalog_manager.cc:5484] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.764231 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 58311afeda3d408f9a387c6ec2398e3c (table default.Table_3bc8fca3155b4f5a982e9eb561b67c35 [id=401e087f4afc46eda0d0f47b661c7f51]). Expected version 1 got 0
I20240503 02:09:17.764411 28656 tablet_bootstrap.cc:654] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.765480 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2834e40454ef48429c31b50da5d2ab0f
I20240503 02:09:17.765825 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 2834e40454ef48429c31b50da5d2ab0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.765923 28654 tablet_replica.cc:330] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.765951 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 2834e40454ef48429c31b50da5d2ab0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.765985 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 48620839bd8d4bbd9f655c1b8aa180af
I20240503 02:09:17.766000 28654 raft_consensus.cc:2229] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.766041 28661 tablet_replica.cc:330] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.766053 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 48620839bd8d4bbd9f655c1b8aa180af deleted and replaced by tablet 8b7a955c305c4ac3a6ecd9248943a618
I20240503 02:09:17.766058 28654 raft_consensus.cc:2258] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.766089 28661 raft_consensus.cc:2229] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.766095 29566 tablet.cc:1722] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.766117 28661 raft_consensus.cc:2258] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.766371 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7a955c305c4ac3a6ecd9248943a618) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.766942 29566 tablet.cc:1722] T 58311afeda3d408f9a387c6ec2398e3c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.767079 28655 tablet_bootstrap.cc:492] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.767117 28655 ts_tablet_manager.cc:1367] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.767208 28655 raft_consensus.cc:348] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.767232 28654 ts_tablet_manager.cc:1875] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.767261 28655 raft_consensus.cc:374] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.767279 28655 raft_consensus.cc:729] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.767334 28655 consensus_queue.cc:260] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.767364 28410 raft_consensus.cc:482] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.767400 28410 raft_consensus.cc:504] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.767432 28655 ts_tablet_manager.cc:1398] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.767485 28655 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.767483 28410 leader_election.cc:288] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.767810 28655 tablet_bootstrap.cc:654] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.768193 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a559ab04d7c4b148c054656cc3863db" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.768289 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7a955c305c4ac3a6ecd9248943a618) from {username='slave'} at 127.0.0.1:43176
W20240503 02:09:17.768448 27523 leader_election.cc:341] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.768680 28654 ts_tablet_manager.cc:1888] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.768725 28654 log.cc:1196] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2834e40454ef48429c31b50da5d2ab0f
I20240503 02:09:17.768754 28664 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.768792 28664 ts_tablet_manager.cc:1367] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:17.768947 28654 ts_tablet_manager.cc:1909] T 2834e40454ef48429c31b50da5d2ab0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.769104 28654 tablet_replica.cc:330] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.769152 28654 raft_consensus.cc:2229] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.769179 28654 raft_consensus.cc:2258] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.769717 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.769752 28664 raft_consensus.cc:348] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.769819 28664 raft_consensus.cc:374] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.769836 28664 raft_consensus.cc:729] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.769829 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ca84293d2cac408591b688cc2b45a5f2" } modify_external_catalogs: true
I20240503 02:09:17.769868 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ca84293d2cac408591b688cc2b45a5f2" } modify_external_catalogs: true
I20240503 02:09:17.769869 28664 consensus_queue.cc:260] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.770013 28664 ts_tablet_manager.cc:1398] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:17.770071 28664 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.770280 28654 ts_tablet_manager.cc:1875] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.771193 28664 tablet_bootstrap.cc:654] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.771462 28654 ts_tablet_manager.cc:1888] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.771504 28654 log.cc:1196] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/48620839bd8d4bbd9f655c1b8aa180af
I20240503 02:09:17.771654 28654 ts_tablet_manager.cc:1909] T 48620839bd8d4bbd9f655c1b8aa180af P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.771761 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a559ab04d7c4b148c054656cc3863db" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:17.771924 27524 leader_election.cc:341] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.771967 27524 leader_election.cc:302] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.772131 30097 tablet.cc:1722] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.772190 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 48620839bd8d4bbd9f655c1b8aa180af (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:17.772378 28507 raft_consensus.cc:2726] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.772485 28656 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.772521 28656 ts_tablet_manager.cc:1367] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:17.772634 28656 raft_consensus.cc:348] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.772680 28656 raft_consensus.cc:374] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.772696 28656 raft_consensus.cc:729] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.772728 28656 consensus_queue.cc:260] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.772800 28656 ts_tablet_manager.cc:1398] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.772873 28656 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.772975 27201 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.773211 28656 tablet_bootstrap.cc:654] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.773221 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 58311afeda3d408f9a387c6ec2398e3c (table default.Table_3bc8fca3155b4f5a982e9eb561b67c35 [id=401e087f4afc46eda0d0f47b661c7f51]). Expected version 1 got 0
I20240503 02:09:17.773582 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.773672 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7a955c305c4ac3a6ecd9248943a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.775940 27419 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.776043 27419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7a955c305c4ac3a6ecd9248943a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.776024 30101 tablet.cc:1722] T 58311afeda3d408f9a387c6ec2398e3c P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.776137 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet e0b2d6d7518f44ba855a43086f24de4d (table default.Table_9cb244f396f642349217ee1381639223 [id=4a014d7616de49d99d5225b5a5c5c964]). Expected version 1 got 0
I20240503 02:09:17.777171 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7a955c305c4ac3a6ecd9248943a618) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.777843 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 2834e40454ef48429c31b50da5d2ab0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.777925 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 4d471aedbe2a45369909f9590e3dfb83 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.778184 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ff2349951a8b4f709892c40103b7cdf1
I20240503 02:09:17.778386 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.778481 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7a955c305c4ac3a6ecd9248943a618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.779637 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2349951a8b4f709892c40103b7cdf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.779799 28710 tablet.cc:1722] T e0b2d6d7518f44ba855a43086f24de4d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.779847 28654 tablet_replica.cc:330] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.779902 28654 raft_consensus.cc:2229] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.779930 28654 raft_consensus.cc:2258] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.780025 28654 ts_tablet_manager.cc:1875] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.781176 28654 ts_tablet_manager.cc:1888] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.781216 28654 log.cc:1196] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ff2349951a8b4f709892c40103b7cdf1
I20240503 02:09:17.781430 28654 ts_tablet_manager.cc:1909] T ff2349951a8b4f709892c40103b7cdf1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
W20240503 02:09:17.781427 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 48620839bd8d4bbd9f655c1b8aa180af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48620839bd8d4bbd9f655c1b8aa180af already in progress: creating tablet
W20240503 02:09:17.781476 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 2834e40454ef48429c31b50da5d2ab0f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2834e40454ef48429c31b50da5d2ab0f already in progress: creating tablet
I20240503 02:09:17.781535 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d86e6ae97444426d955f743e6842a150 [id=7a6c26f3de2541ebb9d9b8275a66c5a6] alter complete (version 2)
I20240503 02:09:17.781845 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2349951a8b4f709892c40103b7cdf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.781912 28653 tablet_replica.cc:330] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.781960 28653 raft_consensus.cc:2229] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.781991 28653 raft_consensus.cc:2258] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.782083 28653 ts_tablet_manager.cc:1875] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.782433 28409 raft_consensus.cc:482] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.782478 28409 raft_consensus.cc:504] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.782577 28409 leader_election.cc:288] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.782754 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d471aedbe2a45369909f9590e3dfb83" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.782816 27719 raft_consensus.cc:2454] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.783067 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3bc8fca3155b4f5a982e9eb561b67c35 [id=401e087f4afc46eda0d0f47b661c7f51] alter complete (version 1)
I20240503 02:09:17.783216 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ff2349951a8b4f709892c40103b7cdf1 (table default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]) successfully deleted
I20240503 02:09:17.783373 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4d471aedbe2a45369909f9590e3dfb83
I20240503 02:09:17.783397 28653 ts_tablet_manager.cc:1888] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.783437 28653 log.cc:1196] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ff2349951a8b4f709892c40103b7cdf1
I20240503 02:09:17.783463 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 4d471aedbe2a45369909f9590e3dfb83 deleted and replaced by tablet f2f62c59d2e24782ba20fb8d4f8bec03
I20240503 02:09:17.783612 28653 ts_tablet_manager.cc:1909] T ff2349951a8b4f709892c40103b7cdf1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.783789 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 with delete_type TABLET_DATA_DELETED (replaced by tablet f2f62c59d2e24782ba20fb8d4f8bec03) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.783882 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 with delete_type TABLET_DATA_DELETED (replaced by tablet f2f62c59d2e24782ba20fb8d4f8bec03) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.783869 28654 tablet_replica.cc:330] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.783967 28654 raft_consensus.cc:2229] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.783998 28654 raft_consensus.cc:2258] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.784318 28654 ts_tablet_manager.cc:1875] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.784482 27388 leader_election.cc:302] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.784551 28409 raft_consensus.cc:2781] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.784572 28409 raft_consensus.cc:482] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.784588 28409 raft_consensus.cc:3037] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.785176 28409 raft_consensus.cc:504] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.785276 28409 leader_election.cc:288] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.785396 28654 ts_tablet_manager.cc:1888] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.785435 28654 log.cc:1196] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4d471aedbe2a45369909f9590e3dfb83
I20240503 02:09:17.785460 28507 raft_consensus.cc:482] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.785503 28507 raft_consensus.cc:504] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.785573 28654 ts_tablet_manager.cc:1909] T 4d471aedbe2a45369909f9590e3dfb83 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.785589 28507 leader_election.cc:288] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.785722 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d2fdc79fec4c60803eb4f9edb792e5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.785813 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d471aedbe2a45369909f9590e3dfb83" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.785863 27718 raft_consensus.cc:3037] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.786495 27718 raft_consensus.cc:2454] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.785804 27719 raft_consensus.cc:2454] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.786782 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:17.786957 27523 leader_election.cc:302] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.786962 27388 leader_election.cc:302] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.787024 28766 raft_consensus.cc:2781] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.787030 28507 raft_consensus.cc:2781] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.787055 28507 raft_consensus.cc:482] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.787065 28766 raft_consensus.cc:686] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.787072 28507 raft_consensus.cc:3037] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.787102 28766 consensus_queue.cc:237] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.787747 28507 raft_consensus.cc:504] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.787853 28507 leader_election.cc:288] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.788162 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d2fdc79fec4c60803eb4f9edb792e5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.788220 27719 raft_consensus.cc:3037] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.788798 27719 raft_consensus.cc:2454] T 32d2fdc79fec4c60803eb4f9edb792e5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.785809 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2349951a8b4f709892c40103b7cdf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.789134 27523 leader_election.cc:302] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.789168 28655 tablet_bootstrap.cc:492] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.789211 28655 ts_tablet_manager.cc:1367] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20240503 02:09:17.789382 28655 raft_consensus.cc:348] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.789438 28655 raft_consensus.cc:374] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.789454 28655 raft_consensus.cc:729] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.789487 28655 consensus_queue.cc:260] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.789566 28655 ts_tablet_manager.cc:1398] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.789618 28655 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.789950 28655 tablet_bootstrap.cc:654] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.790294 28507 raft_consensus.cc:2781] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.790292 28740 raft_consensus.cc:482] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.790344 28507 raft_consensus.cc:686] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.790340 28740 raft_consensus.cc:504] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.790380 28507 consensus_queue.cc:237] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.790429 28740 leader_election.cc:288] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.790823 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2acfdf916c40878a51a85eaadf7dbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.790884 27451 raft_consensus.cc:2454] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.791285 27659 leader_election.cc:302] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.791358 28740 raft_consensus.cc:2781] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.791350 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ff2349951a8b4f709892c40103b7cdf1 (table default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]) successfully deleted
I20240503 02:09:17.791383 28740 raft_consensus.cc:482] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.791401 28740 raft_consensus.cc:3037] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.791586 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d471aedbe2a45369909f9590e3dfb83 with delete_type TABLET_DATA_DELETED (replaced by tablet f2f62c59d2e24782ba20fb8d4f8bec03) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.791808 28661 ts_tablet_manager.cc:1875] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.791997 28740 raft_consensus.cc:504] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.792094 28740 leader_election.cc:288] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.792191 28653 tablet_replica.cc:330] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.792445 28653 raft_consensus.cc:2229] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.792502 28653 pending_rounds.cc:62] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:17.792518 28653 pending_rounds.cc:68] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420038296154112 op_type: NO_OP noop_request { }
I20240503 02:09:17.792549 28653 raft_consensus.cc:2866] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.792568 28653 raft_consensus.cc:2258] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.792901 28766 raft_consensus.cc:482] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.792935 28766 raft_consensus.cc:504] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.792979 27702 tablet_service.cc:1467] Processing CreateTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.793032 28766 leader_election.cc:288] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.793071 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f62c59d2e24782ba20fb8d4f8bec03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.792842 27562 tablet_service.cc:1467] Processing CreateTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.793200 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f62c59d2e24782ba20fb8d4f8bec03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.794369 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d587597b0e4243ad98a7f20a1043f5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.794442 27719 raft_consensus.cc:2454] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.794545 27388 leader_election.cc:302] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.794798 28661 ts_tablet_manager.cc:1888] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.794811 27848 consensus_queue.cc:1035] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.794847 28661 log.cc:1196] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2834e40454ef48429c31b50da5d2ab0f
I20240503 02:09:17.795020 28661 ts_tablet_manager.cc:1909] T 2834e40454ef48429c31b50da5d2ab0f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.795156 28766 raft_consensus.cc:2781] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.795186 28766 raft_consensus.cc:482] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.795204 28766 raft_consensus.cc:3037] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.795787 28766 raft_consensus.cc:504] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.795805 28656 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.795851 28656 ts_tablet_manager.cc:1367] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240503 02:09:17.795838 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab2acfdf916c40878a51a85eaadf7dbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.795886 28766 leader_election.cc:288] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.795984 28656 raft_consensus.cc:348] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.796036 28656 raft_consensus.cc:374] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.796056 28656 raft_consensus.cc:729] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.796090 28656 consensus_queue.cc:260] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.796171 28656 ts_tablet_manager.cc:1398] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.796231 28656 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.796558 28656 tablet_bootstrap.cc:654] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.796845 27195 catalog_manager.cc:5484] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.797288 27425 tablet_service.cc:1467] Processing CreateTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (DEFAULT_TABLE table=default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.797422 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f62c59d2e24782ba20fb8d4f8bec03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.797453 28661 tablet_replica.cc:330] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.797459 27201 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9cb244f396f642349217ee1381639223 [id=4a014d7616de49d99d5225b5a5c5c964] alter complete (version 1)
I20240503 02:09:17.797511 28661 raft_consensus.cc:2229] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.797537 28661 raft_consensus.cc:2258] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.797659 28710 tablet.cc:1722] T 6873697085ec4322a5871dd591dd81dd P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.797797 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.798290 28661 ts_tablet_manager.cc:1875] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.799401 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d587597b0e4243ad98a7f20a1043f5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.799465 27718 raft_consensus.cc:3037] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.799511 28661 ts_tablet_manager.cc:1888] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.799556 28661 log.cc:1196] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/48620839bd8d4bbd9f655c1b8aa180af
I20240503 02:09:17.799696 28661 ts_tablet_manager.cc:1909] T 48620839bd8d4bbd9f655c1b8aa180af P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.799710 28664 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.799741 28664 ts_tablet_manager.cc:1367] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240503 02:09:17.799868 28664 raft_consensus.cc:348] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.799917 28664 raft_consensus.cc:374] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.799932 28664 raft_consensus.cc:729] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.799968 28664 consensus_queue.cc:260] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.800019 27718 raft_consensus.cc:2454] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.800045 28664 ts_tablet_manager.cc:1398] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.800046 28661 tablet_replica.cc:330] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.800108 28664 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.800127 28661 raft_consensus.cc:2229] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.800151 28661 raft_consensus.cc:2258] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.800428 28664 tablet_bootstrap.cc:654] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.800788 27388 leader_election.cc:302] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.800912 28409 raft_consensus.cc:2781] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.800946 28409 raft_consensus.cc:686] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.800977 28409 consensus_queue.cc:237] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.801693 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 48620839bd8d4bbd9f655c1b8aa180af (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:17.801786 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 3297f72c695944debdc8c733280bb86a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.802194 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet ab2acfdf916c40878a51a85eaadf7dbf from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.802246 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 427f0063e66e4688be5c455dfdfe6e90 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.803014 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3297f72c695944debdc8c733280bb86a
I20240503 02:09:17.803093 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 3297f72c695944debdc8c733280bb86a deleted and replaced by tablet 3979ea6b19f443d1a96deefb8acb7867
I20240503 02:09:17.803364 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 3297f72c695944debdc8c733280bb86a with delete_type TABLET_DATA_DELETED (replaced by tablet 3979ea6b19f443d1a96deefb8acb7867) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.803432 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 3297f72c695944debdc8c733280bb86a with delete_type TABLET_DATA_DELETED (replaced by tablet 3979ea6b19f443d1a96deefb8acb7867) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.803520 28654 tablet_replica.cc:330] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.803582 28654 raft_consensus.cc:2229] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.795893 27451 raft_consensus.cc:3037] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.803642 28654 raft_consensus.cc:2258] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.803916 28654 ts_tablet_manager.cc:1875] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.804209 27451 raft_consensus.cc:2454] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.804349 27659 leader_election.cc:302] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.804431 28740 raft_consensus.cc:2781] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.804446 28661 ts_tablet_manager.cc:1875] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.804497 28740 raft_consensus.cc:686] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.804572 28740 consensus_queue.cc:237] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.805095 28654 ts_tablet_manager.cc:1888] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.805137 28654 log.cc:1196] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3297f72c695944debdc8c733280bb86a
W20240503 02:09:17.805166 30104 consensus_queue.cc:1159] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "bd4b79bd8b57443faa6fa5e464570188" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20240503 02:09:17.805313 28654 ts_tablet_manager.cc:1909] T 3297f72c695944debdc8c733280bb86a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.805698 28655 tablet_bootstrap.cc:492] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.805699 28661 ts_tablet_manager.cc:1888] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.805737 28655 ts_tablet_manager.cc:1367] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240503 02:09:17.805743 28661 log.cc:1196] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4d471aedbe2a45369909f9590e3dfb83
I20240503 02:09:17.805879 28655 raft_consensus.cc:348] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.805925 28655 raft_consensus.cc:374] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.805943 28655 raft_consensus.cc:729] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.805977 28655 consensus_queue.cc:260] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.805992 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 427f0063e66e4688be5c455dfdfe6e90
I20240503 02:09:17.806053 28655 ts_tablet_manager.cc:1398] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.806067 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 427f0063e66e4688be5c455dfdfe6e90 deleted and replaced by tablet 5d79bb60500d483985daa3ab36380409
I20240503 02:09:17.806113 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.806119 28655 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.806142 27197 catalog_manager.cc:6554] ReplaceTablet: tablet ab2acfdf916c40878a51a85eaadf7dbf deleted and replaced by tablet 2fd1cb352c40442696afd2fc0bcac0fd
I20240503 02:09:17.806260 28661 ts_tablet_manager.cc:1909] T 4d471aedbe2a45369909f9590e3dfb83 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.806344 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf with delete_type TABLET_DATA_DELETED (replaced by tablet 2fd1cb352c40442696afd2fc0bcac0fd) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.806488 28661 tablet_replica.cc:330] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.806491 28655 tablet_bootstrap.cc:654] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.806488 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3297f72c695944debdc8c733280bb86a (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]) successfully deleted
I20240503 02:09:17.806535 28661 raft_consensus.cc:2229] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.806605 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 with delete_type TABLET_DATA_DELETED (replaced by tablet 5d79bb60500d483985daa3ab36380409) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.806622 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:17.806674 28661 raft_consensus.cc:2258] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.806741 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf with delete_type TABLET_DATA_DELETED (replaced by tablet 2fd1cb352c40442696afd2fc0bcac0fd) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.806773 28661 ts_tablet_manager.cc:1875] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.807544 27562 tablet_service.cc:1514] Processing DeleteTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 with delete_type TABLET_DATA_DELETED (replaced by tablet 5d79bb60500d483985daa3ab36380409) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.807936 28654 tablet_replica.cc:330] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.808004 28654 raft_consensus.cc:2229] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.808032 28654 raft_consensus.cc:2258] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.808218 28661 ts_tablet_manager.cc:1888] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.808259 28661 log.cc:1196] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ff2349951a8b4f709892c40103b7cdf1
I20240503 02:09:17.808436 28661 ts_tablet_manager.cc:1909] T ff2349951a8b4f709892c40103b7cdf1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.808465 28656 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.808487 28654 ts_tablet_manager.cc:1875] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.808501 28656 ts_tablet_manager.cc:1367] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:17.808594 28656 raft_consensus.cc:348] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.808637 28656 raft_consensus.cc:374] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.808653 28656 raft_consensus.cc:729] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.808651 28661 tablet_replica.cc:330] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.808686 28656 consensus_queue.cc:260] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.808703 28661 raft_consensus.cc:2229] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.808732 28661 raft_consensus.cc:2258] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.808770 28656 ts_tablet_manager.cc:1398] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.808898 28656 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.809221 28661 ts_tablet_manager.cc:1875] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.809581 28654 ts_tablet_manager.cc:1888] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.809612 28654 log.cc:1196] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.809710 28654 ts_tablet_manager.cc:1909] T ab2acfdf916c40878a51a85eaadf7dbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.809849 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ff2349951a8b4f709892c40103b7cdf1 (table default.Table_ca84293d2cac408591b688cc2b45a5f2 [id=62dd607ddf83411d9ca041648a24ef47]) successfully deleted
I20240503 02:09:17.810464 28661 ts_tablet_manager.cc:1888] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.810503 28661 log.cc:1196] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3297f72c695944debdc8c733280bb86a
I20240503 02:09:17.810468 27848 consensus_queue.cc:1035] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.810607 28656 tablet_bootstrap.cc:654] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.810678 28661 ts_tablet_manager.cc:1909] T 3297f72c695944debdc8c733280bb86a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.810874 28653 ts_tablet_manager.cc:1875] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.811021 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 3979ea6b19f443d1a96deefb8acb7867 (DEFAULT_TABLE table=default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.811044 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd1cb352c40442696afd2fc0bcac0fd (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.811108 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd1cb352c40442696afd2fc0bcac0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.811192 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3979ea6b19f443d1a96deefb8acb7867. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.812093 28653 ts_tablet_manager.cc:1888] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.812135 28653 log.cc:1196] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4d471aedbe2a45369909f9590e3dfb83
I20240503 02:09:17.812222 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 5d79bb60500d483985daa3ab36380409 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.812289 28661 tablet_replica.cc:330] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.812304 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d79bb60500d483985daa3ab36380409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.812337 28661 raft_consensus.cc:2229] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.812351 28653 ts_tablet_manager.cc:1909] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.812426 28661 pending_rounds.cc:62] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:17.812449 28661 pending_rounds.cc:68] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420038367858688 op_type: NO_OP noop_request { }
I20240503 02:09:17.812477 28661 raft_consensus.cc:2866] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.812494 28661 raft_consensus.cc:2258] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.812968 28664 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.813005 28664 ts_tablet_manager.cc:1367] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:17.813156 28655 tablet_bootstrap.cc:492] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.813153 28664 raft_consensus.cc:348] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.813196 28655 ts_tablet_manager.cc:1367] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:17.813201 28664 raft_consensus.cc:374] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.813220 28664 raft_consensus.cc:729] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.813256 28664 consensus_queue.cc:260] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.813333 28655 raft_consensus.cc:348] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.813376 28664 ts_tablet_manager.cc:1398] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.813387 28655 raft_consensus.cc:374] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.813406 28655 raft_consensus.cc:729] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.813431 28664 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.813433 28655 consensus_queue.cc:260] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.813467 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3297f72c695944debdc8c733280bb86a (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]) successfully deleted
I20240503 02:09:17.813511 28655 ts_tablet_manager.cc:1398] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.813563 28655 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.813670 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 5d79bb60500d483985daa3ab36380409 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.813760 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d79bb60500d483985daa3ab36380409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.813767 28661 ts_tablet_manager.cc:1875] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.813912 28664 tablet_bootstrap.cc:654] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.813969 28655 tablet_bootstrap.cc:654] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.814976 27562 tablet_service.cc:1467] Processing CreateTablet for tablet 3979ea6b19f443d1a96deefb8acb7867 (DEFAULT_TABLE table=default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.815060 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3979ea6b19f443d1a96deefb8acb7867. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.816118 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd1cb352c40442696afd2fc0bcac0fd (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.816242 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd1cb352c40442696afd2fc0bcac0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.817067 28655 tablet_bootstrap.cc:492] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.817103 28655 ts_tablet_manager.cc:1367] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.817229 28655 raft_consensus.cc:348] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.817281 28655 raft_consensus.cc:374] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.817298 28655 raft_consensus.cc:729] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.817373 28655 consensus_queue.cc:260] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.817466 28655 ts_tablet_manager.cc:1398] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.817664 28655 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.818009 28655 tablet_bootstrap.cc:654] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.818003 27195 catalog_manager.cc:5484] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.818099 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.818491 28661 ts_tablet_manager.cc:1888] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.818532 28661 log.cc:1196] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.818702 28661 ts_tablet_manager.cc:1909] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.818923 28664 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.818964 28664 ts_tablet_manager.cc:1367] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:17.819075 28664 raft_consensus.cc:348] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.819119 28664 raft_consensus.cc:374] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.819136 28664 raft_consensus.cc:729] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.819160 28664 consensus_queue.cc:260] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.819236 28664 ts_tablet_manager.cc:1398] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.819275 28661 tablet_replica.cc:330] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.819319 28661 raft_consensus.cc:2229] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.819342 28661 raft_consensus.cc:2258] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.819572 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ab2acfdf916c40878a51a85eaadf7dbf (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
I20240503 02:09:17.819281 28664 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.820160 28664 tablet_bootstrap.cc:654] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.820225 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4d471aedbe2a45369909f9590e3dfb83 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:17.820560 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 3297f72c695944debdc8c733280bb86a with delete_type TABLET_DATA_DELETED (replaced by tablet 3979ea6b19f443d1a96deefb8acb7867) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.820626 27419 tablet_service.cc:1514] Processing DeleteTablet for tablet ab2acfdf916c40878a51a85eaadf7dbf with delete_type TABLET_DATA_DELETED (replaced by tablet 2fd1cb352c40442696afd2fc0bcac0fd) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.820654 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 3979ea6b19f443d1a96deefb8acb7867 (DEFAULT_TABLE table=default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.820683 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 5d79bb60500d483985daa3ab36380409 (DEFAULT_TABLE table=default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.820735 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd1cb352c40442696afd2fc0bcac0fd (DEFAULT_TABLE table=default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.820765 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3979ea6b19f443d1a96deefb8acb7867. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.820811 27433 tablet_service.cc:1514] Processing DeleteTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 with delete_type TABLET_DATA_DELETED (replaced by tablet 5d79bb60500d483985daa3ab36380409) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.820770 28655 tablet_bootstrap.cc:492] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.821031 28655 ts_tablet_manager.cc:1367] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.821138 28655 raft_consensus.cc:348] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.821184 28655 raft_consensus.cc:374] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.821200 28655 raft_consensus.cc:729] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.821233 28655 consensus_queue.cc:260] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.821877 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 2834e40454ef48429c31b50da5d2ab0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.822077 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd1cb352c40442696afd2fc0bcac0fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.822444 28655 ts_tablet_manager.cc:1398] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.822504 28655 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.822804 28655 tablet_bootstrap.cc:654] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.823581 28653 tablet_replica.cc:330] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.823642 28653 raft_consensus.cc:2229] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.823678 28653 raft_consensus.cc:2258] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.823830 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d79bb60500d483985daa3ab36380409. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.824474 28661 ts_tablet_manager.cc:1875] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.826421 28661 ts_tablet_manager.cc:1888] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.826467 28653 ts_tablet_manager.cc:1875] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.826515 28661 log.cc:1196] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/427f0063e66e4688be5c455dfdfe6e90
I20240503 02:09:17.826668 28661 ts_tablet_manager.cc:1909] T 427f0063e66e4688be5c455dfdfe6e90 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.826786 28656 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.826833 28656 ts_tablet_manager.cc:1367] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.000s
I20240503 02:09:17.826941 28664 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.826925 28656 raft_consensus.cc:348] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.826973 28656 raft_consensus.cc:374] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.826972 28664 ts_tablet_manager.cc:1367] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:17.826990 28656 raft_consensus.cc:729] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.827028 28656 consensus_queue.cc:260] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.827111 28656 ts_tablet_manager.cc:1398] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.827107 28664 raft_consensus.cc:348] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.827157 28664 raft_consensus.cc:374] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.827162 28656 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.827174 28664 raft_consensus.cc:729] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.827207 28664 consensus_queue.cc:260] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.827286 28664 ts_tablet_manager.cc:1398] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.827353 28664 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.827378 28656 tablet_bootstrap.cc:654] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.827692 28664 tablet_bootstrap.cc:654] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.827804 28653 ts_tablet_manager.cc:1888] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.827848 28653 log.cc:1196] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3297f72c695944debdc8c733280bb86a
I20240503 02:09:17.828032 28653 ts_tablet_manager.cc:1909] T 3297f72c695944debdc8c733280bb86a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.828725 28653 tablet_replica.cc:330] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.828783 28653 raft_consensus.cc:2229] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.828805 28653 raft_consensus.cc:2258] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.828967 28653 ts_tablet_manager.cc:1875] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.829262 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3297f72c695944debdc8c733280bb86a (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]) successfully deleted
I20240503 02:09:17.829355 28766 consensus_queue.cc:1035] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.830098 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 48620839bd8d4bbd9f655c1b8aa180af with delete_type TABLET_DATA_DELETED (replaced by tablet 8b7a955c305c4ac3a6ecd9248943a618) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.830194 28740 raft_consensus.cc:482] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.830233 28740 raft_consensus.cc:504] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.830328 28740 leader_election.cc:288] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.830333 28653 ts_tablet_manager.cc:1888] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.830374 28653 log.cc:1196] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.830471 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36072c70fb464c9786bf4fef8f285acb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:17.830603 27659 leader_election.cc:341] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.830816 28653 ts_tablet_manager.cc:1909] T ab2acfdf916c40878a51a85eaadf7dbf P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.831012 28653 tablet_replica.cc:330] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.831058 28653 raft_consensus.cc:2229] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.831084 28653 raft_consensus.cc:2258] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.831260 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ab2acfdf916c40878a51a85eaadf7dbf (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
W20240503 02:09:17.831404 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 427f0063e66e4688be5c455dfdfe6e90 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 427f0063e66e4688be5c455dfdfe6e90 already in progress: creating tablet
I20240503 02:09:17.831435 28653 ts_tablet_manager.cc:1875] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.831746 28654 tablet_replica.cc:330] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.831810 28654 raft_consensus.cc:2229] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.831837 28654 raft_consensus.cc:2258] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.832008 28654 ts_tablet_manager.cc:1875] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.832893 28656 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.832929 28656 ts_tablet_manager.cc:1367] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.833037 28656 raft_consensus.cc:348] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.833086 28656 raft_consensus.cc:374] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.833103 28656 raft_consensus.cc:729] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.833127 28656 consensus_queue.cc:260] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.833232 28656 ts_tablet_manager.cc:1398] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.833302 28656 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.833508 28653 ts_tablet_manager.cc:1888] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.833549 28653 log.cc:1196] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2834e40454ef48429c31b50da5d2ab0f
I20240503 02:09:17.833633 28656 tablet_bootstrap.cc:654] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.833683 28653 ts_tablet_manager.cc:1909] T 2834e40454ef48429c31b50da5d2ab0f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.833869 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ab2acfdf916c40878a51a85eaadf7dbf (table default.Table_5706cd992b4448de941e51cf8f3a22ca [id=5a61a46ebb574aa2a30b9764a0a4a18d]) successfully deleted
I20240503 02:09:17.833956 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 427f0063e66e4688be5c455dfdfe6e90 (table default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]) successfully deleted
I20240503 02:09:17.834014 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b737687622d24ad89da74d6418ce396e" } modify_external_catalogs: true
I20240503 02:09:17.834059 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b737687622d24ad89da74d6418ce396e" } modify_external_catalogs: true
I20240503 02:09:17.834044 28655 tablet_bootstrap.cc:492] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.834045 28653 tablet_replica.cc:330] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.834105 28655 ts_tablet_manager.cc:1367] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:17.834134 28653 raft_consensus.cc:2229] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.834160 28653 raft_consensus.cc:2258] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.834224 28655 raft_consensus.cc:348] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.834270 28655 raft_consensus.cc:374] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.834285 28655 raft_consensus.cc:729] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.834327 28655 consensus_queue.cc:260] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.834429 28655 ts_tablet_manager.cc:1398] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.834488 28655 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.834785 28655 tablet_bootstrap.cc:654] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.835249 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2834e40454ef48429c31b50da5d2ab0f (table default.Table_e7f335e7089a44c9aff8bc2ddf88f113 [id=d682b02864cd4fa6b10469ce13cbbf2f]) successfully deleted
I20240503 02:09:17.835453 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet efefb11948624bdfb610a14f81c8c976
I20240503 02:09:17.835525 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
W20240503 02:09:17.835580 27195 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet f2f62c59d2e24782ba20fb8d4f8bec03 consensus state has no leader
I20240503 02:09:17.835638 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet efefb11948624bdfb610a14f81c8c976 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.835793 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet efefb11948624bdfb610a14f81c8c976 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.835853 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet efefb11948624bdfb610a14f81c8c976 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.835947 28653 ts_tablet_manager.cc:1875] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.836817 28661 tablet_replica.cc:330] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.836994 28661 raft_consensus.cc:2229] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.837028 28661 raft_consensus.cc:2258] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.837157 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.837518 28661 ts_tablet_manager.cc:1875] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.837728 28653 ts_tablet_manager.cc:1888] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.837766 28653 log.cc:1196] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/48620839bd8d4bbd9f655c1b8aa180af
I20240503 02:09:17.837896 28653 ts_tablet_manager.cc:1909] T 48620839bd8d4bbd9f655c1b8aa180af P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.838176 28664 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.838220 28664 ts_tablet_manager.cc:1367] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:17.838336 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d2fdc79fec4c60803eb4f9edb792e5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.838358 28664 raft_consensus.cc:348] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.838388 27450 raft_consensus.cc:3037] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.838404 28664 raft_consensus.cc:374] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.838421 28664 raft_consensus.cc:729] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.838455 28664 consensus_queue.cc:260] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.838490 28507 raft_consensus.cc:482] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.838547 28664 ts_tablet_manager.cc:1398] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.838534 28507 raft_consensus.cc:504] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.838603 28664 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.838624 28507 leader_election.cc:288] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.838917 28664 tablet_bootstrap.cc:654] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.838981 27450 raft_consensus.cc:2454] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.839162 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32d2fdc79fec4c60803eb4f9edb792e5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.839159 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36072c70fb464c9786bf4fef8f285acb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.839215 27451 raft_consensus.cc:2362] T 32d2fdc79fec4c60803eb4f9edb792e5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:17.839221 27583 raft_consensus.cc:2454] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.839270 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.839319 27718 raft_consensus.cc:2454] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.839531 28653 tablet_replica.cc:330] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.839584 28653 raft_consensus.cc:2229] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.839609 28653 raft_consensus.cc:2258] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.839695 27523 leader_election.cc:302] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.839756 28653 ts_tablet_manager.cc:1875] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.839776 28656 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.839808 28661 ts_tablet_manager.cc:1888] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.839825 28656 ts_tablet_manager.cc:1367] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:17.839850 28661 log.cc:1196] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/efefb11948624bdfb610a14f81c8c976
I20240503 02:09:17.839967 28656 raft_consensus.cc:348] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.840016 28656 raft_consensus.cc:374] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.840034 28656 raft_consensus.cc:729] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.840057 28661 ts_tablet_manager.cc:1909] T efefb11948624bdfb610a14f81c8c976 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.840068 28656 consensus_queue.cc:260] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.840152 28656 ts_tablet_manager.cc:1398] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.840198 28656 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.840289 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet efefb11948624bdfb610a14f81c8c976 (table default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]) successfully deleted
I20240503 02:09:17.840489 28656 tablet_bootstrap.cc:654] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.840615 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d587597b0e4243ad98a7f20a1043f5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.840688 27588 raft_consensus.cc:2454] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.840750 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:17.840775 27656 leader_election.cc:341] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.840626 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d587597b0e4243ad98a7f20a1043f5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.840859 30009 tablet.cc:1722] T e0b2d6d7518f44ba855a43086f24de4d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.841009 27656 leader_election.cc:302] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
W20240503 02:09:17.841135 27656 leader_election.cc:341] T ab2acfdf916c40878a51a85eaadf7dbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: ab2acfdf916c40878a51a85eaadf7dbf
I20240503 02:09:17.841429 28693 raft_consensus.cc:2781] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.841466 28693 raft_consensus.cc:482] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.841483 28693 raft_consensus.cc:3037] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:17.841500 27386 leader_election.cc:341] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 4d471aedbe2a45369909f9590e3dfb83
W20240503 02:09:17.841674 27386 leader_election.cc:341] T 4d471aedbe2a45369909f9590e3dfb83 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 4d471aedbe2a45369909f9590e3dfb83
W20240503 02:09:17.841710 27524 leader_election.cc:341] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.841871 28766 consensus_queue.cc:1035] T 53b8f4c16c99410286f69fef3236e920 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.841979 28710 tablet.cc:1722] T 58311afeda3d408f9a387c6ec2398e3c P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.842058 28693 raft_consensus.cc:504] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.842154 28693 leader_election.cc:288] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.842232 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 58311afeda3d408f9a387c6ec2398e3c (table default.Table_3bc8fca3155b4f5a982e9eb561b67c35 [id=401e087f4afc46eda0d0f47b661c7f51]). Expected version 1 got 0
I20240503 02:09:17.842325 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 48620839bd8d4bbd9f655c1b8aa180af (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:17.839782 29241 raft_consensus.cc:2781] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.842551 29241 raft_consensus.cc:482] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.842569 29241 raft_consensus.cc:3037] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.843160 29241 raft_consensus.cc:504] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.843250 28653 ts_tablet_manager.cc:1888] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.843264 29241 leader_election.cc:288] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.843286 28653 log.cc:1196] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/efefb11948624bdfb610a14f81c8c976
I20240503 02:09:17.843438 28653 ts_tablet_manager.cc:1909] T efefb11948624bdfb610a14f81c8c976 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.843432 28654 ts_tablet_manager.cc:1888] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.844070 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet efefb11948624bdfb610a14f81c8c976 (table default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]) successfully deleted
I20240503 02:09:17.844070 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.844173 28655 tablet_bootstrap.cc:492] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.844210 28655 ts_tablet_manager.cc:1367] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:17.843480 28654 log.cc:1196] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/427f0063e66e4688be5c455dfdfe6e90
W20240503 02:09:17.844228 27524 leader_election.cc:341] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.844339 28655 raft_consensus.cc:348] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.844390 28655 raft_consensus.cc:374] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.844408 28655 raft_consensus.cc:729] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.844450 28655 consensus_queue.cc:260] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.844537 28655 ts_tablet_manager.cc:1398] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.844585 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_fe12e659d715492e91da868574f70963"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.844702 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe12e659d715492e91da868574f70963 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.844843 28655 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.840873 27587 raft_consensus.cc:3037] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.845178 28655 tablet_bootstrap.cc:654] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.845304 28654 ts_tablet_manager.cc:1909] T 427f0063e66e4688be5c455dfdfe6e90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.845531 27587 raft_consensus.cc:2454] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.846319 28664 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.846356 28664 ts_tablet_manager.cc:1367] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:17.846498 28664 raft_consensus.cc:348] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.846549 28664 raft_consensus.cc:374] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.846566 28664 raft_consensus.cc:729] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.846599 28664 consensus_queue.cc:260] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.846680 28664 ts_tablet_manager.cc:1398] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.846730 28664 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.847079 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.847126 28664 tablet_bootstrap.cc:654] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.847139 27718 raft_consensus.cc:3037] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.840494 28507 consensus_queue.cc:1035] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20240503 02:09:17.847756 27718 raft_consensus.cc:2454] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.847934 28766 raft_consensus.cc:482] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.847983 28766 raft_consensus.cc:504] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.848065 28766 leader_election.cc:288] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.848137 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 95d96daa3de7403a865d872e8d678628 (DEFAULT_TABLE table=default.Table_fe12e659d715492e91da868574f70963 [id=26590db4e4c146de8e025b1e892a8a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.848237 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95d96daa3de7403a865d872e8d678628. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.848354 28655 tablet_bootstrap.cc:492] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.848392 28655 ts_tablet_manager.cc:1367] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.848513 28655 raft_consensus.cc:348] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.848563 28655 raft_consensus.cc:374] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.848582 28655 raft_consensus.cc:729] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.848605 28655 consensus_queue.cc:260] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.848685 28655 ts_tablet_manager.cc:1398] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.848738 28655 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.848763 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 95d96daa3de7403a865d872e8d678628 (DEFAULT_TABLE table=default.Table_fe12e659d715492e91da868574f70963 [id=26590db4e4c146de8e025b1e892a8a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.848841 28656 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.848851 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95d96daa3de7403a865d872e8d678628. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.848877 28656 ts_tablet_manager.cc:1367] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:17.848986 28656 raft_consensus.cc:348] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.849033 28656 raft_consensus.cc:374] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.849051 28656 raft_consensus.cc:729] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.849053 28655 tablet_bootstrap.cc:654] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.849077 28656 consensus_queue.cc:260] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.849164 28656 ts_tablet_manager.cc:1398] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.849223 28656 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.849618 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 95d96daa3de7403a865d872e8d678628 (DEFAULT_TABLE table=default.Table_fe12e659d715492e91da868574f70963 [id=26590db4e4c146de8e025b1e892a8a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.849704 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95d96daa3de7403a865d872e8d678628. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.849706 28656 tablet_bootstrap.cc:654] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.850080 28766 raft_consensus.cc:482] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.850121 28766 raft_consensus.cc:504] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.850203 28766 leader_election.cc:288] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.850216 28664 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.850253 28664 ts_tablet_manager.cc:1367] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.850373 28664 raft_consensus.cc:348] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.850417 28664 raft_consensus.cc:374] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.850437 28664 raft_consensus.cc:729] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.850462 28664 consensus_queue.cc:260] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.850557 28664 ts_tablet_manager.cc:1398] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.850613 28664 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.850813 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36072c70fb464c9786bf4fef8f285acb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.850914 28664 tablet_bootstrap.cc:654] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:17.850984 27659 leader_election.cc:341] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.851877 27523 leader_election.cc:302] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.851897 28655 tablet_bootstrap.cc:492] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.851872 27717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b262a1decf457ebab85ea106273fe2" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.851933 28655 ts_tablet_manager.cc:1367] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.851946 27717 raft_consensus.cc:2454] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.852048 28655 raft_consensus.cc:348] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.852098 28655 raft_consensus.cc:374] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.852115 28655 raft_consensus.cc:729] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.852147 28655 consensus_queue.cc:260] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.852169 29241 raft_consensus.cc:2781] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.852214 29241 raft_consensus.cc:686] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.852242 28655 ts_tablet_manager.cc:1398] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.852247 29241 consensus_queue.cc:237] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.852288 28655 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.852329 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.852424 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7078864d79114bbe910112efb8ca01c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.852478 27718 raft_consensus.cc:2454] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.852612 28655 tablet_bootstrap.cc:654] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.853061 27388 leader_election.cc:302] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.853143 27388 leader_election.cc:302] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.853192 28766 raft_consensus.cc:2781] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.853215 28766 raft_consensus.cc:482] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.853231 28766 raft_consensus.cc:3037] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.853358 28767 raft_consensus.cc:2781] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.853397 28767 raft_consensus.cc:482] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.853415 28767 raft_consensus.cc:3037] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.853650 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7078864d79114bbe910112efb8ca01c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.853701 27587 raft_consensus.cc:2454] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.853874 28766 raft_consensus.cc:504] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.853971 28766 leader_election.cc:288] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.853999 28767 raft_consensus.cc:504] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.854084 28767 leader_election.cc:288] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.854154 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36072c70fb464c9786bf4fef8f285acb" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.854195 27582 raft_consensus.cc:3037] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.854352 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7078864d79114bbe910112efb8ca01c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.854408 27586 raft_consensus.cc:3037] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.854552 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b262a1decf457ebab85ea106273fe2" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.854601 27587 raft_consensus.cc:3037] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.854642 27717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7078864d79114bbe910112efb8ca01c3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.854694 27717 raft_consensus.cc:3037] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.854770 27582 raft_consensus.cc:2454] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.854831 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b262a1decf457ebab85ea106273fe2" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.855033 27586 raft_consensus.cc:2454] T 7078864d79114bbe910112efb8ca01c3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.855161 27587 raft_consensus.cc:2454] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.855325 27717 raft_consensus.cc:2454] T 7078864d79114bbe910112efb8ca01c3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.855332 28655 tablet_bootstrap.cc:492] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.855367 28655 ts_tablet_manager.cc:1367] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.855479 28655 raft_consensus.cc:348] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.855523 28655 raft_consensus.cc:374] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.855540 28655 raft_consensus.cc:729] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.855531 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b262a1decf457ebab85ea106273fe2" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.855587 27718 raft_consensus.cc:3037] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.855576 28655 consensus_queue.cc:260] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.855652 28655 ts_tablet_manager.cc:1398] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.855697 28655 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.856096 28655 tablet_bootstrap.cc:654] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.856173 27718 raft_consensus.cc:2454] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.856668 27656 leader_election.cc:302] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:17.856825 28774 raft_consensus.cc:2781] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.856871 28774 raft_consensus.cc:686] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.856905 28774 consensus_queue.cc:237] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.856920 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dfed422989eb4f6684bf55a5ec489606" } modify_external_catalogs: true
I20240503 02:09:17.856978 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_dfed422989eb4f6684bf55a5ec489606" } modify_external_catalogs: true
I20240503 02:09:17.857364 27386 leader_election.cc:302] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.857457 27386 leader_election.cc:302] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.857525 28409 raft_consensus.cc:2781] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.857570 28409 raft_consensus.cc:686] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.857610 28409 consensus_queue.cc:237] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.857810 28409 raft_consensus.cc:2781] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.857847 28409 raft_consensus.cc:686] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:17.857877 28409 consensus_queue.cc:237] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.858788 28655 tablet_bootstrap.cc:492] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.858829 28655 ts_tablet_manager.cc:1367] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:17.858939 28655 raft_consensus.cc:348] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.858996 28655 raft_consensus.cc:374] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.859015 28655 raft_consensus.cc:729] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.859041 28655 consensus_queue.cc:260] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.859117 28655 ts_tablet_manager.cc:1398] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.859162 28655 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.859279 27195 catalog_manager.cc:5484] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.859376 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.859416 27195 catalog_manager.cc:5484] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.859472 28655 tablet_bootstrap.cc:654] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.860119 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f471bbd597ef4763bf54eac9fb7ccc9a
I20240503 02:09:17.860443 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.860462 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.860741 28656 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.860782 28656 ts_tablet_manager.cc:1367] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240503 02:09:17.860862 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet f471bbd597ef4763bf54eac9fb7ccc9a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.860886 28656 raft_consensus.cc:348] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.860944 28656 raft_consensus.cc:374] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.860960 28656 raft_consensus.cc:729] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.860992 28656 consensus_queue.cc:260] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.861068 28656 ts_tablet_manager.cc:1398] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.861127 28661 tablet_replica.cc:330] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.861191 28661 raft_consensus.cc:2229] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.861225 28661 raft_consensus.cc:2258] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.861222 28653 tablet_replica.cc:330] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.861277 28653 raft_consensus.cc:2229] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.861342 28653 raft_consensus.cc:2258] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.861529 28653 ts_tablet_manager.cc:1875] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.861778 28656 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.862102 28656 tablet_bootstrap.cc:654] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.862576 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 427f0063e66e4688be5c455dfdfe6e90 with delete_type TABLET_DATA_DELETED (replaced by tablet 5d79bb60500d483985daa3ab36380409) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.862670 27195 catalog_manager.cc:5484] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.862762 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:17.862861 28653 ts_tablet_manager.cc:1888] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.862900 28653 log.cc:1196] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f471bbd597ef4763bf54eac9fb7ccc9a
I20240503 02:09:17.863070 28653 ts_tablet_manager.cc:1909] T f471bbd597ef4763bf54eac9fb7ccc9a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.863145 28661 ts_tablet_manager.cc:1875] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.863462 28653 tablet_replica.cc:330] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.863517 28653 raft_consensus.cc:2229] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.863548 28653 raft_consensus.cc:2258] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.863828 28653 ts_tablet_manager.cc:1875] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.863986 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f471bbd597ef4763bf54eac9fb7ccc9a (table default.Table_dfed422989eb4f6684bf55a5ec489606 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:17.864296 28656 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.864334 28656 ts_tablet_manager.cc:1367] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:17.864423 28661 ts_tablet_manager.cc:1888] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.864465 28661 log.cc:1196] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f471bbd597ef4763bf54eac9fb7ccc9a
I20240503 02:09:17.864459 28656 raft_consensus.cc:348] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.864511 28656 raft_consensus.cc:374] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.864531 28656 raft_consensus.cc:729] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.864567 28656 consensus_queue.cc:260] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:17.864630 27195 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3979ea6b19f443d1a96deefb8acb7867 consensus state has no leader
I20240503 02:09:17.864638 28661 ts_tablet_manager.cc:1909] T f471bbd597ef4763bf54eac9fb7ccc9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.864647 28656 ts_tablet_manager.cc:1398] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.864931 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.864977 27197 catalog_manager.cc:5484] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:17.865152 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f471bbd597ef4763bf54eac9fb7ccc9a (table default.Table_dfed422989eb4f6684bf55a5ec489606 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:17.865164 28653 ts_tablet_manager.cc:1888] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.865203 28653 log.cc:1196] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/427f0063e66e4688be5c455dfdfe6e90
I20240503 02:09:17.865365 28653 ts_tablet_manager.cc:1909] T 427f0063e66e4688be5c455dfdfe6e90 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.865666 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 427f0063e66e4688be5c455dfdfe6e90 (table default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]) successfully deleted
I20240503 02:09:17.865677 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1c35f5fed2ec4213839722d452f35c49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.865808 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c35f5fed2ec4213839722d452f35c49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.867807 28766 consensus_queue.cc:1035] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.868227 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 427f0063e66e4688be5c455dfdfe6e90 (table default.Table_b148a22d0440457daff31b9cf7d84da0 [id=1c179ca2f2044670a2937eac21eb7624]) successfully deleted
I20240503 02:09:17.868232 28654 tablet_replica.cc:330] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.868311 28654 raft_consensus.cc:2229] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.868341 28654 raft_consensus.cc:2258] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.870424 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 26ff4d9b25b546ee99946ccc34c22669 (DEFAULT_TABLE table=default.Table_1c35f5fed2ec4213839722d452f35c49 [id=f7544fedad004884b71b9b6cdaa4a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.870520 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ff4d9b25b546ee99946ccc34c22669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.870781 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3acc4505a58544e983dddb84ffdceac1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.870885 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3acc4505a58544e983dddb84ffdceac1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.871356 28655 tablet_bootstrap.cc:492] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.871393 28655 ts_tablet_manager.cc:1367] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:17.871434 29558 tablet.cc:1722] T e0b2d6d7518f44ba855a43086f24de4d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.871516 28655 raft_consensus.cc:348] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.871564 28655 raft_consensus.cc:374] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.871579 28655 raft_consensus.cc:729] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.871614 28655 consensus_queue.cc:260] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.871690 28655 ts_tablet_manager.cc:1398] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.871739 28655 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.871932 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 26ff4d9b25b546ee99946ccc34c22669 (DEFAULT_TABLE table=default.Table_1c35f5fed2ec4213839722d452f35c49 [id=f7544fedad004884b71b9b6cdaa4a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.872025 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ff4d9b25b546ee99946ccc34c22669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.872083 28655 tablet_bootstrap.cc:654] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.872876 28409 consensus_queue.cc:1035] T 640547cbbff2492489dff09073733982 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.873119 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 26ff4d9b25b546ee99946ccc34c22669 (DEFAULT_TABLE table=default.Table_1c35f5fed2ec4213839722d452f35c49 [id=f7544fedad004884b71b9b6cdaa4a649]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.873210 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ff4d9b25b546ee99946ccc34c22669. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.873215 28656 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.875231 29241 consensus_queue.cc:1035] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.877390 28656 tablet_bootstrap.cc:654] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.877946 28655 tablet_bootstrap.cc:492] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.877988 28655 ts_tablet_manager.cc:1367] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.878120 28655 raft_consensus.cc:348] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.878170 28655 raft_consensus.cc:374] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.878185 28655 raft_consensus.cc:729] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.878218 28655 consensus_queue.cc:260] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.878304 28655 ts_tablet_manager.cc:1398] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.878350 28655 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.878613 27565 tablet_service.cc:1467] Processing CreateTablet for tablet f4b168c17ac44030a73596e9d79123a7 (DEFAULT_TABLE table=default.Table_3acc4505a58544e983dddb84ffdceac1 [id=40a840fc5b274a579372794fd3a3a60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.878675 28655 tablet_bootstrap.cc:654] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.878707 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b168c17ac44030a73596e9d79123a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.878827 27694 tablet_service.cc:1467] Processing CreateTablet for tablet f4b168c17ac44030a73596e9d79123a7 (DEFAULT_TABLE table=default.Table_3acc4505a58544e983dddb84ffdceac1 [id=40a840fc5b274a579372794fd3a3a60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.878908 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b168c17ac44030a73596e9d79123a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.879153 28655 tablet_bootstrap.cc:492] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.879191 28655 ts_tablet_manager.cc:1367] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.879402 28655 raft_consensus.cc:348] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.879456 28655 raft_consensus.cc:374] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.879473 28655 raft_consensus.cc:729] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.879510 28655 consensus_queue.cc:260] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.879590 28655 ts_tablet_manager.cc:1398] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.879640 28655 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.879995 28655 tablet_bootstrap.cc:654] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.880630 27427 tablet_service.cc:1467] Processing CreateTablet for tablet f4b168c17ac44030a73596e9d79123a7 (DEFAULT_TABLE table=default.Table_3acc4505a58544e983dddb84ffdceac1 [id=40a840fc5b274a579372794fd3a3a60d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.880774 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b168c17ac44030a73596e9d79123a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.882027 28655 tablet_bootstrap.cc:492] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.882071 28655 ts_tablet_manager.cc:1367] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:17.882200 28655 raft_consensus.cc:348] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.882251 28655 raft_consensus.cc:374] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.882266 28655 raft_consensus.cc:729] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.882297 28655 consensus_queue.cc:260] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.882395 28655 ts_tablet_manager.cc:1398] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.882555 28655 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.882680 27848 consensus_queue.cc:1035] T 04ffc308a3b44eed8dce0a99d8f0a6fa P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.883126 28655 tablet_bootstrap.cc:654] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.884812 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.885489 28654 ts_tablet_manager.cc:1875] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.886896 28654 ts_tablet_manager.cc:1888] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.888031 28664 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.888080 28664 ts_tablet_manager.cc:1367] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.000s
I20240503 02:09:17.888190 28664 raft_consensus.cc:348] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.888236 28664 raft_consensus.cc:374] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.888253 28664 raft_consensus.cc:729] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.888289 28664 consensus_queue.cc:260] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.888370 28664 ts_tablet_manager.cc:1398] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.888438 28664 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.888760 28664 tablet_bootstrap.cc:654] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.888643 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_0a9f9b8dbd214ba69b58a7dd43c93f7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.888825 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a9f9b8dbd214ba69b58a7dd43c93f7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.888631 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bb6cee84b5624c2c93af79f09ea9d32e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.889251 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb6cee84b5624c2c93af79f09ea9d32e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.892992 28774 raft_consensus.cc:482] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.893038 28774 raft_consensus.cc:504] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.893117 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 647ba8fd2b324e8884e22e16ca414471 (DEFAULT_TABLE table=default.Table_0a9f9b8dbd214ba69b58a7dd43c93f7b [id=b5ff8c5028a642cebcb7a14702574265]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.893128 28774 leader_election.cc:288] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.893218 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647ba8fd2b324e8884e22e16ca414471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.893481 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 647ba8fd2b324e8884e22e16ca414471 (DEFAULT_TABLE table=default.Table_0a9f9b8dbd214ba69b58a7dd43c93f7b [id=b5ff8c5028a642cebcb7a14702574265]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.893564 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647ba8fd2b324e8884e22e16ca414471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.893929 28655 tablet_bootstrap.cc:492] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.893962 28655 ts_tablet_manager.cc:1367] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.000s
I20240503 02:09:17.894089 28655 raft_consensus.cc:348] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.894140 28655 raft_consensus.cc:374] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.894156 28655 raft_consensus.cc:729] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.894192 28655 consensus_queue.cc:260] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.894272 28655 ts_tablet_manager.cc:1398] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.894346 28655 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.894651 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7a955c305c4ac3a6ecd9248943a618" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.894658 28655 tablet_bootstrap.cc:654] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.894709 27448 raft_consensus.cc:2454] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.894966 27659 leader_election.cc:302] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.895040 28774 raft_consensus.cc:2781] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.895061 28774 raft_consensus.cc:482] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.895078 28774 raft_consensus.cc:3037] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.895663 28774 raft_consensus.cc:504] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.895776 28774 leader_election.cc:288] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.896410 28664 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.896451 28664 ts_tablet_manager.cc:1367] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:17.896536 28664 raft_consensus.cc:348] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.896586 28664 raft_consensus.cc:374] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.896601 28664 raft_consensus.cc:729] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.896633 28664 consensus_queue.cc:260] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.896710 28664 ts_tablet_manager.cc:1398] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.896773 28664 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.896947 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f42523eb93df4875ac58a242ed2d93a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:17.896997 28664 tablet_bootstrap.cc:654] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:17.897043 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f42523eb93df4875ac58a242ed2d93a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.897156 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7a955c305c4ac3a6ecd9248943a618" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.897224 27447 raft_consensus.cc:3037] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.897876 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_63bcae89d666400790b5517d39e533ca" } new_table_name: "default.Table_9019f5ee81ad45399f17053162dc0be8" modify_external_catalogs: true
I20240503 02:09:17.897922 27447 raft_consensus.cc:2454] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.898561 27201 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:17.899314 27659 leader_election.cc:302] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.899426 28693 raft_consensus.cc:2781] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.899478 28693 raft_consensus.cc:686] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.899516 28693 consensus_queue.cc:237] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.899567 27694 tablet_service.cc:1467] Processing CreateTablet for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3 (DEFAULT_TABLE table=default.Table_bb6cee84b5624c2c93af79f09ea9d32e [id=a26f902c47ff4389af95416eadbddfd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.899641 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.901054 27427 tablet_service.cc:1467] Processing CreateTablet for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3 (DEFAULT_TABLE table=default.Table_bb6cee84b5624c2c93af79f09ea9d32e [id=a26f902c47ff4389af95416eadbddfd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.901146 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.902096 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 9d08a51420834ca0afffd14e8743e1c5 (DEFAULT_TABLE table=default.Table_f42523eb93df4875ac58a242ed2d93a5 [id=a7566949ac244087ae89f85cb0f6fa79]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.902182 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d08a51420834ca0afffd14e8743e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.902366 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 9d08a51420834ca0afffd14e8743e1c5 (DEFAULT_TABLE table=default.Table_f42523eb93df4875ac58a242ed2d93a5 [id=a7566949ac244087ae89f85cb0f6fa79]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.902469 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d08a51420834ca0afffd14e8743e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.903941 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_6cffc813b4ed451a91ad4fa95b35f4f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.904026 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.904645 27195 catalog_manager.cc:5484] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.907184 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_3a6a4426c6844434bffe8904438b4408"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.907282 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a6a4426c6844434bffe8904438b4408 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.907316 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed2da494a334cbab2f24ca578adc494 (DEFAULT_TABLE table=default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.907413 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed2da494a334cbab2f24ca578adc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.907327 28740 consensus_queue.cc:1035] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:09:17.909224 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed2da494a334cbab2f24ca578adc494 (DEFAULT_TABLE table=default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.909453 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed2da494a334cbab2f24ca578adc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.909794 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_f1584265111f4e44a06af54179a2f451"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.909881 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1584265111f4e44a06af54179a2f451 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.912595 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b6c9984b7488438a863dcf791b5b794f (DEFAULT_TABLE table=default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.912734 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c9984b7488438a863dcf791b5b794f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.912829 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b6c9984b7488438a863dcf791b5b794f (DEFAULT_TABLE table=default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.912917 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c9984b7488438a863dcf791b5b794f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.888245 28656 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.913642 28656 ts_tablet_manager.cc:1367] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20240503 02:09:17.913758 28656 raft_consensus.cc:348] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.913808 28656 raft_consensus.cc:374] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.913820 28656 raft_consensus.cc:729] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.913841 28656 consensus_queue.cc:260] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.913941 28656 ts_tablet_manager.cc:1398] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.914016 28656 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.914271 28656 tablet_bootstrap.cc:654] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.889537 29241 consensus_queue.cc:1035] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.893260 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7a955c305c4ac3a6ecd9248943a618" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.914535 27587 raft_consensus.cc:2454] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.914739 28654 log.cc:1196] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/efefb11948624bdfb610a14f81c8c976
I20240503 02:09:17.896270 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7a955c305c4ac3a6ecd9248943a618" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.914857 27586 raft_consensus.cc:3037] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.914988 28656 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.915037 28656 ts_tablet_manager.cc:1367] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:17.915123 28656 raft_consensus.cc:348] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.915169 28656 raft_consensus.cc:374] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.915184 28656 raft_consensus.cc:729] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.915217 28656 consensus_queue.cc:260] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.915313 28656 ts_tablet_manager.cc:1398] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.915522 27586 raft_consensus.cc:2454] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.896948 27562 tablet_service.cc:1467] Processing CreateTablet for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3 (DEFAULT_TABLE table=default.Table_bb6cee84b5624c2c93af79f09ea9d32e [id=a26f902c47ff4389af95416eadbddfd5]), partition=RANGE (key) PARTITION UNBOUNDED
W20240503 02:09:17.900204 27524 consensus_peers.cc:485] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Couldn't send request to peer bd4b79bd8b57443faa6fa5e464570188. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f471bbd597ef4763bf54eac9fb7ccc9a. This is attempt 1: this message will repeat every 5th retry.
I20240503 02:09:17.915647 27562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.901881 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 9d08a51420834ca0afffd14e8743e1c5 (DEFAULT_TABLE table=default.Table_f42523eb93df4875ac58a242ed2d93a5 [id=a7566949ac244087ae89f85cb0f6fa79]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.915927 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d08a51420834ca0afffd14e8743e1c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.916326 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7 (DEFAULT_TABLE table=default.Table_f1584265111f4e44a06af54179a2f451 [id=9bae714f426143d08eb50b1e1a0bcd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.916409 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.916777 28656 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.908457 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed2da494a334cbab2f24ca578adc494 (DEFAULT_TABLE table=default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.917016 28656 tablet_bootstrap.cc:654] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.917060 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed2da494a334cbab2f24ca578adc494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.912649 27554 tablet_service.cc:1467] Processing CreateTablet for tablet b6c9984b7488438a863dcf791b5b794f (DEFAULT_TABLE table=default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.917425 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c9984b7488438a863dcf791b5b794f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.918720 28654 ts_tablet_manager.cc:1909] T efefb11948624bdfb610a14f81c8c976 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.918943 28654 tablet_replica.cc:330] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.919011 28654 raft_consensus.cc:2229] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.919011 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet efefb11948624bdfb610a14f81c8c976 (table default.Table_b737687622d24ad89da74d6418ce396e [id=f0f675217d1d4260a70f74596f972e6e]) successfully deleted
I20240503 02:09:17.919076 28654 raft_consensus.cc:2258] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.919247 28654 ts_tablet_manager.cc:1875] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.919521 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d35c89a1c8454483aacc3d4449a5afed" } modify_external_catalogs: true
I20240503 02:09:17.919581 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d35c89a1c8454483aacc3d4449a5afed" } modify_external_catalogs: true
I20240503 02:09:17.920135 29558 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.920688 30009 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.920794 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_b461ce136ff84b40a82e638866255964" } new_table_name: "default.Table_6f1cfab914c2494a97e80c38bef7a4ab" modify_external_catalogs: true
I20240503 02:09:17.921023 28654 ts_tablet_manager.cc:1888] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:17.921103 28654 log.cc:1196] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f471bbd597ef4763bf54eac9fb7ccc9a
I20240503 02:09:17.921304 28654 ts_tablet_manager.cc:1909] T f471bbd597ef4763bf54eac9fb7ccc9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.921453 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 36072c70fb464c9786bf4fef8f285acb
I20240503 02:09:17.921667 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 36072c70fb464c9786bf4fef8f285acb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.921667 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7 (DEFAULT_TABLE table=default.Table_f1584265111f4e44a06af54179a2f451 [id=9bae714f426143d08eb50b1e1a0bcd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.921739 28661 tablet_replica.cc:330] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.921756 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.921768 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f471bbd597ef4763bf54eac9fb7ccc9a (table default.Table_dfed422989eb4f6684bf55a5ec489606 [id=a56811ee47c84a1f9e43ed591962b726]) successfully deleted
I20240503 02:09:17.921792 28661 raft_consensus.cc:2229] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.921833 28661 pending_rounds.cc:62] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:17.921849 28661 pending_rounds.cc:68] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420038582108160 op_type: NO_OP noop_request { }
I20240503 02:09:17.921878 28661 raft_consensus.cc:2866] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:17.921907 28661 raft_consensus.cc:2258] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.922181 28661 ts_tablet_manager.cc:1875] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.922946 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 36072c70fb464c9786bf4fef8f285acb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.923060 28654 tablet_replica.cc:330] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.893267 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 647ba8fd2b324e8884e22e16ca414471 (DEFAULT_TABLE table=default.Table_0a9f9b8dbd214ba69b58a7dd43c93f7b [id=b5ff8c5028a642cebcb7a14702574265]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.923327 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647ba8fd2b324e8884e22e16ca414471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.923213 28654 raft_consensus.cc:2229] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.923620 28654 raft_consensus.cc:2258] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.923686 28661 ts_tablet_manager.cc:1888] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.923724 28661 log.cc:1196] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/36072c70fb464c9786bf4fef8f285acb
I20240503 02:09:17.923786 28654 ts_tablet_manager.cc:1875] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.923900 28661 ts_tablet_manager.cc:1909] T 36072c70fb464c9786bf4fef8f285acb P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.924161 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 36072c70fb464c9786bf4fef8f285acb (table default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]) successfully deleted
I20240503 02:09:17.924953 28654 ts_tablet_manager.cc:1888] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.925005 28654 log.cc:1196] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/36072c70fb464c9786bf4fef8f285acb
I20240503 02:09:17.925144 28654 ts_tablet_manager.cc:1909] T 36072c70fb464c9786bf4fef8f285acb P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.925348 27719 raft_consensus.cc:1261] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.925459 27450 raft_consensus.cc:1261] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.925437 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e1f44d51698843e5ac496d080a17ea04"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:17.925530 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1f44d51698843e5ac496d080a17ea04 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:17.925809 29241 consensus_queue.cc:1035] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.926074 29241 consensus_queue.cc:1035] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.926095 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7 (DEFAULT_TABLE table=default.Table_f1584265111f4e44a06af54179a2f451 [id=9bae714f426143d08eb50b1e1a0bcd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.926182 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59aa5f0d43d14cf8bdafc8bb7fedb0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.927520 29566 tablet.cc:1722] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.928682 29241 raft_consensus.cc:482] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.928723 29241 raft_consensus.cc:504] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.928817 29241 leader_election.cc:288] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.929292 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 36072c70fb464c9786bf4fef8f285acb (table default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]) successfully deleted
I20240503 02:09:17.929502 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fada3a7eedae4a1c9f2372ea5699020f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.929565 27718 raft_consensus.cc:2454] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.929719 27523 leader_election.cc:302] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.929790 29241 raft_consensus.cc:2781] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.929812 29241 raft_consensus.cc:482] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.929829 29241 raft_consensus.cc:3037] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.930012 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f42c6ab1499d4b058241376b6f211bde (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:17.930408 29241 raft_consensus.cc:504] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.930507 29241 leader_election.cc:288] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.930677 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 029b393711d7465299f648174787dd60 (DEFAULT_TABLE table=default.Table_e1f44d51698843e5ac496d080a17ea04 [id=8f2b09204df14b6db12ea76c63e3a57b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.930771 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 029b393711d7465299f648174787dd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.930831 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fada3a7eedae4a1c9f2372ea5699020f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.930878 27718 raft_consensus.cc:3037] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.931090 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f42c6ab1499d4b058241376b6f211bde (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:17.931370 27718 raft_consensus.cc:2454] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.931713 27523 leader_election.cc:302] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.931789 29241 raft_consensus.cc:2781] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.931821 29241 raft_consensus.cc:686] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.931854 29241 consensus_queue.cc:237] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.931921 28409 consensus_queue.cc:1035] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.932452 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 029b393711d7465299f648174787dd60 (DEFAULT_TABLE table=default.Table_e1f44d51698843e5ac496d080a17ea04 [id=8f2b09204df14b6db12ea76c63e3a57b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.932540 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 029b393711d7465299f648174787dd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.932778 28655 tablet_bootstrap.cc:492] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.932816 28655 ts_tablet_manager.cc:1367] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.000s
I20240503 02:09:17.933019 28655 raft_consensus.cc:348] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.933075 28655 raft_consensus.cc:374] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.933092 28655 raft_consensus.cc:729] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.933130 28655 consensus_queue.cc:260] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.933218 28655 ts_tablet_manager.cc:1398] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.933281 28655 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.933262 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9019f5ee81ad45399f17053162dc0be8 [id=cd9e7541be36427ea8e01bbba147d869] alter complete (version 2)
I20240503 02:09:17.933990 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fada3a7eedae4a1c9f2372ea5699020f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.934056 27453 raft_consensus.cc:2454] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.934105 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fada3a7eedae4a1c9f2372ea5699020f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.934149 27452 raft_consensus.cc:3037] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.934175 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:17.934726 27452 raft_consensus.cc:2454] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.934109 28710 tablet.cc:1722] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.935074 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 36072c70fb464c9786bf4fef8f285acb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.935144 28653 tablet_replica.cc:330] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.935195 28653 raft_consensus.cc:2229] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.935212 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 029b393711d7465299f648174787dd60 (DEFAULT_TABLE table=default.Table_e1f44d51698843e5ac496d080a17ea04 [id=8f2b09204df14b6db12ea76c63e3a57b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.935308 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 029b393711d7465299f648174787dd60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.935580 27195 catalog_manager.cc:5484] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.936339 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_61db889c4c5a45e793a8bf5976fc5eac" } new_table_name: "default.Table_4e9bb0432eae4ac79f6a792c5e49ca76" modify_external_catalogs: true
I20240503 02:09:17.935221 28653 raft_consensus.cc:2258] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.936822 28653 ts_tablet_manager.cc:1875] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.937415 28656 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.937458 28656 ts_tablet_manager.cc:1367] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.000s
I20240503 02:09:17.937542 28656 raft_consensus.cc:348] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.937589 28656 raft_consensus.cc:374] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.937605 28656 raft_consensus.cc:729] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.937635 28656 consensus_queue.cc:260] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.937650 28655 tablet_bootstrap.cc:654] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.937721 28656 ts_tablet_manager.cc:1398] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.937777 28656 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.937999 28656 tablet_bootstrap.cc:654] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.938197 28653 ts_tablet_manager.cc:1888] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:17.938241 28653 log.cc:1196] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/36072c70fb464c9786bf4fef8f285acb
I20240503 02:09:17.938391 28653 ts_tablet_manager.cc:1909] T 36072c70fb464c9786bf4fef8f285acb P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.938899 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f496c1bdb19a424a9ef4d1f772825869" } new_table_name: "default.Table_70d04cdada5745929c09a3480c78f25c" modify_external_catalogs: true
I20240503 02:09:17.939870 29558 tablet.cc:1722] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.939999 28766 consensus_queue.cc:1035] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.941120 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:17.941215 28664 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.941246 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 36072c70fb464c9786bf4fef8f285acb (table default.Table_d35c89a1c8454483aacc3d4449a5afed [id=3411c3f85b2941deb9658a78e3489d98]) successfully deleted
I20240503 02:09:17.941251 28664 ts_tablet_manager.cc:1367] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.044s user 0.000s sys 0.000s
I20240503 02:09:17.941409 28664 raft_consensus.cc:348] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.941465 28664 raft_consensus.cc:374] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.941483 28664 raft_consensus.cc:729] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.941517 28664 consensus_queue.cc:260] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.941619 28664 ts_tablet_manager.cc:1398] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.941669 28664 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.941893 28664 tablet_bootstrap.cc:654] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.942550 28655 tablet_bootstrap.cc:492] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.942591 28655 ts_tablet_manager.cc:1367] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:17.942723 28655 raft_consensus.cc:348] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.942777 28655 raft_consensus.cc:374] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.942795 28655 raft_consensus.cc:729] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.942821 28655 consensus_queue.cc:260] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.942903 28655 ts_tablet_manager.cc:1398] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.942955 28655 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.943281 28655 tablet_bootstrap.cc:654] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.943253 29240 consensus_queue.cc:1035] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.943320 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698] alter complete (version 1)
I20240503 02:09:17.944353 28656 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.944401 28656 ts_tablet_manager.cc:1367] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:17.944514 28656 raft_consensus.cc:348] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.944556 28656 raft_consensus.cc:374] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.944572 28656 raft_consensus.cc:729] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.944604 28656 consensus_queue.cc:260] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.944681 28656 ts_tablet_manager.cc:1398] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.944730 28656 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.944779 28710 tablet.cc:1722] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:17.944948 28740 raft_consensus.cc:482] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.944989 28740 raft_consensus.cc:504] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.945070 28740 leader_election.cc:288] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.945293 28655 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.945281 28740 consensus_queue.cc:1035] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.945364 28655 ts_tablet_manager.cc:1367] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:17.945487 28655 raft_consensus.cc:348] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.945536 28655 raft_consensus.cc:374] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.945552 28655 raft_consensus.cc:729] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.945585 28655 consensus_queue.cc:260] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.945662 28655 ts_tablet_manager.cc:1398] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.945703 28655 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.945937 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f62c59d2e24782ba20fb8d4f8bec03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.945930 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f62c59d2e24782ba20fb8d4f8bec03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.945989 27587 raft_consensus.cc:2454] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.945989 27448 raft_consensus.cc:2454] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:17.946041 28655 tablet_bootstrap.cc:654] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.946071 28656 tablet_bootstrap.cc:654] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.946422 27659 leader_election.cc:302] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.946588 28740 raft_consensus.cc:2781] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.946625 28740 raft_consensus.cc:482] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.946642 28740 raft_consensus.cc:3037] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.946715 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 051d0dfb8e744d0ca7044f0c03835a90 (table default.Table_4e9bb0432eae4ac79f6a792c5e49ca76 [id=ab3d9c3ffb424c999a8b4d897ccc68c3]). Expected version 1 got 0
I20240503 02:09:17.947279 28740 raft_consensus.cc:504] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.947408 28740 leader_election.cc:288] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.948182 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f62c59d2e24782ba20fb8d4f8bec03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.948237 27448 raft_consensus.cc:3037] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.948760 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 051d0dfb8e744d0ca7044f0c03835a90 (table default.Table_4e9bb0432eae4ac79f6a792c5e49ca76 [id=ab3d9c3ffb424c999a8b4d897ccc68c3]). Expected version 1 got 0
I20240503 02:09:17.948848 27448 raft_consensus.cc:2454] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.948998 27659 leader_election.cc:302] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.949064 28740 raft_consensus.cc:2781] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.949096 28740 raft_consensus.cc:686] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:17.949141 28740 consensus_queue.cc:237] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.949539 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f62c59d2e24782ba20fb8d4f8bec03" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.949604 27587 raft_consensus.cc:3037] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.949687 28655 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.949728 28655 ts_tablet_manager.cc:1367] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.949765 29240 raft_consensus.cc:482] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.949800 29240 raft_consensus.cc:504] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.949848 28655 raft_consensus.cc:348] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.949895 28655 raft_consensus.cc:374] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.949893 29240 leader_election.cc:288] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.949913 28655 raft_consensus.cc:729] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.949944 28655 consensus_queue.cc:260] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.950021 28655 ts_tablet_manager.cc:1398] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.950071 28655 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.950214 27587 raft_consensus.cc:2454] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:17.950460 28655 tablet_bootstrap.cc:654] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.950825 27717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca02d87dfd549e9bf01c7f3d38928e3" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.950886 27717 raft_consensus.cc:2454] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.951350 27523 leader_election.cc:302] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.951364 27195 catalog_manager.cc:5484] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.951421 29240 raft_consensus.cc:2781] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.951447 29240 raft_consensus.cc:482] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.951452 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:17.951464 29240 raft_consensus.cc:3037] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.951478 28656 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.951511 28656 ts_tablet_manager.cc:1367] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:17.951632 28656 raft_consensus.cc:348] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.951683 28656 raft_consensus.cc:374] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.951699 28656 raft_consensus.cc:729] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.951735 28656 consensus_queue.cc:260] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.951838 28656 ts_tablet_manager.cc:1398] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.952061 29240 raft_consensus.cc:504] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.952144 28656 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.952148 29240 leader_election.cc:288] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.952527 28656 tablet_bootstrap.cc:654] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.952946 27717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca02d87dfd549e9bf01c7f3d38928e3" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.953004 27717 raft_consensus.cc:3037] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.953668 27717 raft_consensus.cc:2454] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.953941 27523 leader_election.cc:302] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:17.954257 29240 raft_consensus.cc:2781] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:17.954316 29240 raft_consensus.cc:686] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:17.954357 29240 consensus_queue.cc:237] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.954795 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca02d87dfd549e9bf01c7f3d38928e3" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:17.954862 27448 raft_consensus.cc:2454] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:17.954913 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca02d87dfd549e9bf01c7f3d38928e3" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:17.954947 27447 raft_consensus.cc:3037] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.955180 27198 catalog_manager.cc:5484] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:17.955595 27447 raft_consensus.cc:2454] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:17.957433 29558 tablet.cc:1722] T 051d0dfb8e744d0ca7044f0c03835a90 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.957664 28710 tablet.cc:1722] T 051d0dfb8e744d0ca7044f0c03835a90 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.957903 30097 tablet.cc:1722] T 60368acf76cd4e9eb8e6750b37ad95b9 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.958475 28710 tablet.cc:1722] T 60368acf76cd4e9eb8e6750b37ad95b9 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.958643 30009 tablet.cc:1722] T 60368acf76cd4e9eb8e6750b37ad95b9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.959005 29240 consensus_queue.cc:1035] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.959306 29566 tablet.cc:1722] T 051d0dfb8e744d0ca7044f0c03835a90 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:17.961594 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3acf1387019c4a17ba3e98e20e8a90d6" } new_table_name: "default.Table_439bc66b02c44366bffee27082fb5426" modify_external_catalogs: true
I20240503 02:09:17.962577 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_4e9bb0432eae4ac79f6a792c5e49ca76 [id=ab3d9c3ffb424c999a8b4d897ccc68c3] alter complete (version 1)
I20240503 02:09:17.963126 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_70d04cdada5745929c09a3480c78f25c [id=57d220ab05944f938c205ffe5861eaa4] alter complete (version 1)
I20240503 02:09:17.963517 28664 tablet_bootstrap.cc:492] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.963562 28664 ts_tablet_manager.cc:1367] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:17.963558 28655 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.963599 28655 ts_tablet_manager.cc:1367] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:17.963693 28664 raft_consensus.cc:348] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.963743 28664 raft_consensus.cc:374] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.963716 28655 raft_consensus.cc:348] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.963760 28664 raft_consensus.cc:729] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.963765 28655 raft_consensus.cc:374] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.963783 28655 raft_consensus.cc:729] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.963784 28664 consensus_queue.cc:260] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.963816 28655 consensus_queue.cc:260] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.963877 28664 ts_tablet_manager.cc:1398] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.963925 28664 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.963922 28655 ts_tablet_manager.cc:1398] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.963981 28655 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.964201 28664 tablet_bootstrap.cc:654] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.964366 28655 tablet_bootstrap.cc:654] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.965065 28507 raft_consensus.cc:482] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.965116 28507 raft_consensus.cc:504] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.965217 28507 leader_election.cc:288] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:17.965358 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
W20240503 02:09:17.965528 27524 leader_election.cc:341] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.965840 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
W20240503 02:09:17.966213 27523 leader_election.cc:341] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:17.966256 27523 leader_election.cc:302] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:17.966315 28507 raft_consensus.cc:2726] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:17.966562 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_430cab1f60494f67984d34f207740a2b" } modify_external_catalogs: true
I20240503 02:09:17.966639 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_430cab1f60494f67984d34f207740a2b" } modify_external_catalogs: true
I20240503 02:09:17.967515 28664 tablet_bootstrap.cc:492] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.967581 28664 ts_tablet_manager.cc:1367] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.967669 28664 raft_consensus.cc:348] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.967717 28664 raft_consensus.cc:374] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.967733 28664 raft_consensus.cc:729] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.967756 28664 consensus_queue.cc:260] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.967837 28664 ts_tablet_manager.cc:1398] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.967907 28664 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.968194 28664 tablet_bootstrap.cc:654] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.969563 28655 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.969607 28655 ts_tablet_manager.cc:1367] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:17.969739 28655 raft_consensus.cc:348] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.969794 28655 raft_consensus.cc:374] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.969812 28655 raft_consensus.cc:729] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.969847 28655 consensus_queue.cc:260] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.969933 28655 ts_tablet_manager.cc:1398] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.969981 28655 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.970111 28710 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:17.970310 28655 tablet_bootstrap.cc:654] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.971474 29558 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:17.971912 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 44b7bf524ad049dba8cd61f4cde3cec7
I20240503 02:09:17.972362 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.972501 28661 tablet_replica.cc:330] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:17.972492 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.972561 28661 raft_consensus.cc:2229] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.972612 28653 tablet_replica.cc:330] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.972623 28661 raft_consensus.cc:2258] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.972666 28653 raft_consensus.cc:2229] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.972695 28653 raft_consensus.cc:2258] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.973923 28653 ts_tablet_manager.cc:1875] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.974720 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_439bc66b02c44366bffee27082fb5426 [id=c46a0a1c65dd47e38382f2b2d19bff05] alter complete (version 4)
I20240503 02:09:17.974735 27719 raft_consensus.cc:1261] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.975076 28740 consensus_queue.cc:1035] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.975358 28653 ts_tablet_manager.cc:1888] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.975409 28653 log.cc:1196] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/44b7bf524ad049dba8cd61f4cde3cec7
I20240503 02:09:17.975601 28653 ts_tablet_manager.cc:1909] T 44b7bf524ad049dba8cd61f4cde3cec7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.975859 29550 consensus_queue.cc:1035] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.976141 28661 ts_tablet_manager.cc:1875] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.976176 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 44b7bf524ad049dba8cd61f4cde3cec7 (table default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]) successfully deleted
I20240503 02:09:17.972353 30190 tablet.cc:1722] T ed866637e1fa4b9fbcee3bbd93fd785d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:17.977531 28664 tablet_bootstrap.cc:492] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.977571 28664 ts_tablet_manager.cc:1367] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:17.977697 28661 ts_tablet_manager.cc:1888] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.977756 28661 log.cc:1196] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/44b7bf524ad049dba8cd61f4cde3cec7
I20240503 02:09:17.977902 28693 consensus_queue.cc:1035] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.977941 28661 ts_tablet_manager.cc:1909] T 44b7bf524ad049dba8cd61f4cde3cec7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:17.973001 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 44b7bf524ad049dba8cd61f4cde3cec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.976785 28656 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.978255 28656 ts_tablet_manager.cc:1367] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.000s
I20240503 02:09:17.978416 28656 raft_consensus.cc:348] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.978477 28656 raft_consensus.cc:374] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.978493 28656 raft_consensus.cc:729] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.978535 28656 consensus_queue.cc:260] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.978629 28656 ts_tablet_manager.cc:1398] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.978683 28656 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.978791 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e7d8d66c8a284fa69d3f8732a45da2be" } modify_external_catalogs: true
I20240503 02:09:17.978859 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_e7d8d66c8a284fa69d3f8732a45da2be" } modify_external_catalogs: true
I20240503 02:09:17.979013 28656 tablet_bootstrap.cc:654] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.978255 28654 tablet_replica.cc:330] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.979538 28654 raft_consensus.cc:2229] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.979573 28654 raft_consensus.cc:2258] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.975817 27588 raft_consensus.cc:1261] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:17.980003 28774 consensus_queue.cc:1035] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.980294 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 58740c221e9b422d9fcff96e2df8e85e from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.980280 29551 consensus_queue.cc:1035] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.980456 28654 ts_tablet_manager.cc:1875] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.977695 28664 raft_consensus.cc:348] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.981168 28664 raft_consensus.cc:374] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.981186 28664 raft_consensus.cc:729] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.981221 28664 consensus_queue.cc:260] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.981305 28664 ts_tablet_manager.cc:1398] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:17.981402 28664 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.981774 28664 tablet_bootstrap.cc:654] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.982082 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 58e3e21135574e9bba4f1c6896ee545c
I20240503 02:09:17.982328 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e3e21135574e9bba4f1c6896ee545c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.982352 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e3e21135574e9bba4f1c6896ee545c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.982628 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 58740c221e9b422d9fcff96e2df8e85e
I20240503 02:09:17.982719 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 58740c221e9b422d9fcff96e2df8e85e deleted and replaced by tablet 15819b1d3eec4d6fb3d1ef884508370a
I20240503 02:09:17.982865 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 58740c221e9b422d9fcff96e2df8e85e with delete_type TABLET_DATA_DELETED (replaced by tablet 15819b1d3eec4d6fb3d1ef884508370a) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:17.983047 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 58740c221e9b422d9fcff96e2df8e85e with delete_type TABLET_DATA_DELETED (replaced by tablet 15819b1d3eec4d6fb3d1ef884508370a) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:17.985093 28654 ts_tablet_manager.cc:1888] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.985146 28654 log.cc:1196] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/44b7bf524ad049dba8cd61f4cde3cec7
I20240503 02:09:17.985191 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 15819b1d3eec4d6fb3d1ef884508370a (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.985230 28655 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:17.985267 28655 ts_tablet_manager.cc:1367] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20240503 02:09:17.985283 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15819b1d3eec4d6fb3d1ef884508370a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.985375 28654 ts_tablet_manager.cc:1909] T 44b7bf524ad049dba8cd61f4cde3cec7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:17.985415 28655 raft_consensus.cc:348] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.985472 28655 raft_consensus.cc:374] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.985487 28655 raft_consensus.cc:729] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.985512 28655 consensus_queue.cc:260] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.985615 28655 ts_tablet_manager.cc:1398] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20240503 02:09:17.985667 28655 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:17.985849 28693 consensus_queue.cc:1035] T 7282be27904d48f1af33a47693387a5d P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:17.986166 28655 tablet_bootstrap.cc:654] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.986527 28654 tablet_replica.cc:330] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:17.985142 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 15819b1d3eec4d6fb3d1ef884508370a (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.986583 28654 raft_consensus.cc:2229] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:17.986641 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15819b1d3eec4d6fb3d1ef884508370a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.987622 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e3e21135574e9bba4f1c6896ee545c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:17 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.987697 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 58740c221e9b422d9fcff96e2df8e85e with delete_type TABLET_DATA_DELETED (replaced by tablet 15819b1d3eec4d6fb3d1ef884508370a) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.987704 28653 tablet_replica.cc:330] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.987790 28653 raft_consensus.cc:2229] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.987823 28653 raft_consensus.cc:2258] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.988049 28653 ts_tablet_manager.cc:1875] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:17.988220 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 44b7bf524ad049dba8cd61f4cde3cec7 (table default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]) successfully deleted
I20240503 02:09:17.988392 28656 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.988449 28656 ts_tablet_manager.cc:1367] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:17.988597 28656 raft_consensus.cc:348] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.988651 28656 raft_consensus.cc:374] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.988667 28656 raft_consensus.cc:729] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.988699 28656 consensus_queue.cc:260] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.988790 28656 ts_tablet_manager.cc:1398] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.988833 28656 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.989100 28656 tablet_bootstrap.cc:654] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.989478 28653 ts_tablet_manager.cc:1888] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:17.989529 28653 log.cc:1196] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/58e3e21135574e9bba4f1c6896ee545c
I20240503 02:09:17.989707 28653 ts_tablet_manager.cc:1909] T 58e3e21135574e9bba4f1c6896ee545c P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:17.989817 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a166dbdc3574a0183e3cc4896307be2 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:17.989894 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 15819b1d3eec4d6fb3d1ef884508370a (DEFAULT_TABLE table=default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.990016 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15819b1d3eec4d6fb3d1ef884508370a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.990298 28656 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.990343 28656 ts_tablet_manager.cc:1367] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:17.990461 28656 raft_consensus.cc:348] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.990532 28656 raft_consensus.cc:374] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.990551 28656 raft_consensus.cc:729] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.990585 28656 consensus_queue.cc:260] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.990679 28656 ts_tablet_manager.cc:1398] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.990733 28656 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.991050 28656 tablet_bootstrap.cc:654] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.992663 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3a166dbdc3574a0183e3cc4896307be2
I20240503 02:09:17.992714 28656 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.992749 28656 ts_tablet_manager.cc:1367] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:17.992859 28656 raft_consensus.cc:348] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.992887 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.992905 28656 raft_consensus.cc:374] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.992921 28656 raft_consensus.cc:729] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.992925 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 3a166dbdc3574a0183e3cc4896307be2 deleted and replaced by tablet b0e7025bcc084d6d9a6d34e61b75d9c7
I20240503 02:09:17.992954 28656 consensus_queue.cc:260] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.993044 28656 ts_tablet_manager.cc:1398] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.993096 28656 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.993146 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 31b262a1decf457ebab85ea106273fe2 (table default.Table_709fdd1f272c489a815f79b02bfa18b3 [id=fa2cf85615ba4391a016c5051ebf1050]). Expected version 1 got 0
I20240503 02:09:17.993479 28656 tablet_bootstrap.cc:654] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.994163 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 with delete_type TABLET_DATA_DELETED (replaced by tablet b0e7025bcc084d6d9a6d34e61b75d9c7) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:17.994287 29551 raft_consensus.cc:482] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:17.994333 29551 raft_consensus.cc:504] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.994453 29551 leader_election.cc:288] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.994920 28664 tablet_bootstrap.cc:492] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:17.994969 28664 ts_tablet_manager.cc:1367] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:17.995105 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95d96daa3de7403a865d872e8d678628" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:17.995111 28664 raft_consensus.cc:348] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.995172 27588 raft_consensus.cc:2454] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.995180 28664 raft_consensus.cc:374] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.995198 28664 raft_consensus.cc:729] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.995239 28664 consensus_queue.cc:260] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.995333 28664 ts_tablet_manager.cc:1398] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.995422 27386 leader_election.cc:302] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.995586 27557 tablet_service.cc:1467] Processing CreateTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.995648 29551 raft_consensus.cc:2781] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:17.995679 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e7025bcc084d6d9a6d34e61b75d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.995692 29551 raft_consensus.cc:482] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:17.995711 29551 raft_consensus.cc:3037] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.995750 28664 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.996098 28664 tablet_bootstrap.cc:654] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.996369 27694 tablet_service.cc:1467] Processing CreateTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.996377 29551 raft_consensus.cc:504] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:17.996449 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e7025bcc084d6d9a6d34e61b75d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.996481 29551 leader_election.cc:288] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.996840 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95d96daa3de7403a865d872e8d678628" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:17.996893 27588 raft_consensus.cc:3037] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:17.997097 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 58e3e21135574e9bba4f1c6896ee545c (table default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]) successfully deleted
I20240503 02:09:17.997380 28656 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:17.997423 28656 ts_tablet_manager.cc:1367] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:17.997542 28656 raft_consensus.cc:348] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.997606 28656 raft_consensus.cc:374] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:17.997596 27427 tablet_service.cc:1467] Processing CreateTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (DEFAULT_TABLE table=default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:17.997623 28656 raft_consensus.cc:729] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:17.997658 28656 consensus_queue.cc:260] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:17.997735 28656 ts_tablet_manager.cc:1398] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:17.997737 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95d96daa3de7403a865d872e8d678628" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:17.997786 28656 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:17.997766 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95d96daa3de7403a865d872e8d678628" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:17.997797 27720 raft_consensus.cc:2454] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:17.998171 28653 tablet_replica.cc:330] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:17.998239 28653 raft_consensus.cc:2229] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:17.998265 28653 raft_consensus.cc:2258] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:17.998364 28656 tablet_bootstrap.cc:654] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:17.997546 27588 raft_consensus.cc:2454] T 95d96daa3de7403a865d872e8d678628 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:17.999673 27386 leader_election.cc:302] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:17.999758 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0e7025bcc084d6d9a6d34e61b75d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:17.999939 28710 tablet.cc:1722] T 31b262a1decf457ebab85ea106273fe2 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.000519 28664 tablet_bootstrap.cc:492] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.000564 28664 ts_tablet_manager.cc:1367] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.000684 28664 raft_consensus.cc:348] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.000735 28664 raft_consensus.cc:374] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.000751 28664 raft_consensus.cc:729] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.000785 28664 consensus_queue.cc:260] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.000875 28664 ts_tablet_manager.cc:1398] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.000959 28664 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:17.999786 29551 raft_consensus.cc:2781] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.001139 29551 raft_consensus.cc:686] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.001175 29551 consensus_queue.cc:237] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.001379 28664 tablet_bootstrap.cc:654] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.001626 28740 raft_consensus.cc:482] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.001672 28740 raft_consensus.cc:504] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.001778 28740 leader_election.cc:288] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.001899 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ff4d9b25b546ee99946ccc34c22669" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.001960 27582 raft_consensus.cc:2454] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.002070 28653 ts_tablet_manager.cc:1875] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.002067 27656 leader_election.cc:302] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.002132 27853 raft_consensus.cc:2781] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.002156 27853 raft_consensus.cc:482] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.002175 27853 raft_consensus.cc:3037] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.002282 30009 tablet.cc:1722] T 31b262a1decf457ebab85ea106273fe2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.002748 27853 raft_consensus.cc:504] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.002858 27853 leader_election.cc:288] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.003664 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ff4d9b25b546ee99946ccc34c22669" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.003722 27588 raft_consensus.cc:3037] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.004062 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ff4d9b25b546ee99946ccc34c22669" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.004125 27447 raft_consensus.cc:2454] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.004169 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ff4d9b25b546ee99946ccc34c22669" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.004195 27451 raft_consensus.cc:3037] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.004401 27588 raft_consensus.cc:2454] T 26ff4d9b25b546ee99946ccc34c22669 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.004549 27656 leader_election.cc:302] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.004805 27451 raft_consensus.cc:2454] T 26ff4d9b25b546ee99946ccc34c22669 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.004902 29558 tablet.cc:1722] T 31b262a1decf457ebab85ea106273fe2 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.004987 28740 raft_consensus.cc:2781] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.005020 28740 raft_consensus.cc:686] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.005052 28740 consensus_queue.cc:237] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.006214 27198 catalog_manager.cc:5484] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.007866 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet b02d8134f2684fb5a0ddbb9e685c8f79 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.008085 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 with delete_type TABLET_DATA_DELETED (replaced by tablet b0e7025bcc084d6d9a6d34e61b75d9c7) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.008265 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a166dbdc3574a0183e3cc4896307be2 with delete_type TABLET_DATA_DELETED (replaced by tablet b0e7025bcc084d6d9a6d34e61b75d9c7) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.008843 27200 catalog_manager.cc:5484] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.009260 28661 tablet_replica.cc:330] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.009351 28661 raft_consensus.cc:2229] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.009397 28661 raft_consensus.cc:2258] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.009421 27853 raft_consensus.cc:482] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.009479 27853 raft_consensus.cc:504] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.009590 27853 leader_election.cc:288] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.009618 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 44b7bf524ad049dba8cd61f4cde3cec7 (table default.Table_430cab1f60494f67984d34f207740a2b [id=4a13ad9392764c7ba3be948239654999]) successfully deleted
I20240503 02:09:18.009784 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b02d8134f2684fb5a0ddbb9e685c8f79
I20240503 02:09:18.009864 27195 catalog_manager.cc:6554] ReplaceTablet: tablet b02d8134f2684fb5a0ddbb9e685c8f79 deleted and replaced by tablet a24e57083d8f4dd9b450b0c78a611e0f
I20240503 02:09:18.009902 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3979ea6b19f443d1a96deefb8acb7867" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.009971 27588 raft_consensus.cc:2454] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.010097 27656 leader_election.cc:302] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.010157 27853 raft_consensus.cc:2781] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.010179 27853 raft_consensus.cc:482] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.010196 27853 raft_consensus.cc:3037] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.010249 28664 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.010289 28664 ts_tablet_manager.cc:1367] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:18.010439 28664 raft_consensus.cc:348] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.010499 28664 raft_consensus.cc:374] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.010478 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3979ea6b19f443d1a96deefb8acb7867" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.010516 28664 raft_consensus.cc:729] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.010529 27451 raft_consensus.cc:2454] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.010550 28664 consensus_queue.cc:260] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.010603 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 with delete_type TABLET_DATA_DELETED (replaced by tablet a24e57083d8f4dd9b450b0c78a611e0f) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.010653 28664 ts_tablet_manager.cc:1398] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.010675 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 with delete_type TABLET_DATA_DELETED (replaced by tablet a24e57083d8f4dd9b450b0c78a611e0f) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.010725 28664 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.010730 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 5924b7e1ff81434885296e6e3ed6757b from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.010851 27853 raft_consensus.cc:504] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.010963 27853 leader_election.cc:288] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:17.986649 28654 raft_consensus.cc:2258] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.011188 28653 ts_tablet_manager.cc:1888] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.011207 28664 tablet_bootstrap.cc:654] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.011235 28653 log.cc:1196] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/58740c221e9b422d9fcff96e2df8e85e
I20240503 02:09:18.011315 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3979ea6b19f443d1a96deefb8acb7867" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.011392 27588 raft_consensus.cc:3037] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.011442 28653 ts_tablet_manager.cc:1909] T 58740c221e9b422d9fcff96e2df8e85e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.011651 28653 tablet_replica.cc:330] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.011708 28653 raft_consensus.cc:2229] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.011734 28653 raft_consensus.cc:2258] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.011822 28653 ts_tablet_manager.cc:1875] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.011894 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 58740c221e9b422d9fcff96e2df8e85e (table default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]) successfully deleted
I20240503 02:09:18.011962 27588 raft_consensus.cc:2454] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.012288 27656 leader_election.cc:302] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.012391 28693 raft_consensus.cc:2781] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.012442 28693 raft_consensus.cc:686] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.012478 28693 consensus_queue.cc:237] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.013088 28653 ts_tablet_manager.cc:1888] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.013100 28656 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.013132 28653 log.cc:1196] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3a166dbdc3574a0183e3cc4896307be2
I20240503 02:09:18.013139 28656 ts_tablet_manager.cc:1367] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240503 02:09:18.013267 28656 raft_consensus.cc:348] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.013336 28656 raft_consensus.cc:374] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.013312 28653 ts_tablet_manager.cc:1909] T 3a166dbdc3574a0183e3cc4896307be2 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.013357 28656 raft_consensus.cc:729] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.013388 28656 consensus_queue.cc:260] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.013505 28656 ts_tablet_manager.cc:1398] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.013566 28507 raft_consensus.cc:482] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.013607 28507 raft_consensus.cc:504] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.013726 28507 leader_election.cc:288] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.013854 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet fada3a7eedae4a1c9f2372ea5699020f from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.013839 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d08a51420834ca0afffd14e8743e1c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.013900 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3979ea6b19f443d1a96deefb8acb7867" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.013940 27451 raft_consensus.cc:3037] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
W20240503 02:09:18.014190 27524 leader_election.cc:341] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.014506 27451 raft_consensus.cc:2454] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.015058 28654 ts_tablet_manager.cc:1875] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.015375 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d08a51420834ca0afffd14e8743e1c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.015446 27720 raft_consensus.cc:2454] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.015587 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet b02d8134f2684fb5a0ddbb9e685c8f79 with delete_type TABLET_DATA_DELETED (replaced by tablet a24e57083d8f4dd9b450b0c78a611e0f) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.015776 27523 leader_election.cc:302] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.016036 28507 raft_consensus.cc:2781] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.016072 28507 raft_consensus.cc:482] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.016089 28507 raft_consensus.cc:3037] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.016269 27195 catalog_manager.cc:5484] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.016348 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:18.016368 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet fada3a7eedae4a1c9f2372ea5699020f
I20240503 02:09:18.016438 28654 ts_tablet_manager.cc:1888] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.016465 27200 catalog_manager.cc:6554] ReplaceTablet: tablet fada3a7eedae4a1c9f2372ea5699020f deleted and replaced by tablet 0d5f5d7b273241e097bb196744e6f566
I20240503 02:09:18.016475 28654 log.cc:1196] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/58e3e21135574e9bba4f1c6896ee545c
I20240503 02:09:18.016608 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet fada3a7eedae4a1c9f2372ea5699020f with delete_type TABLET_DATA_DELETED (replaced by tablet 0d5f5d7b273241e097bb196744e6f566) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.016660 28654 ts_tablet_manager.cc:1909] T 58e3e21135574e9bba4f1c6896ee545c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.016710 28507 raft_consensus.cc:504] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.016799 28507 leader_election.cc:288] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.017400 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet fada3a7eedae4a1c9f2372ea5699020f with delete_type TABLET_DATA_DELETED (replaced by tablet 0d5f5d7b273241e097bb196744e6f566) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.017416 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d08a51420834ca0afffd14e8743e1c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.017479 27718 raft_consensus.cc:3037] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.017469 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d08a51420834ca0afffd14e8743e1c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.017881 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet fada3a7eedae4a1c9f2372ea5699020f with delete_type TABLET_DATA_DELETED (replaced by tablet 0d5f5d7b273241e097bb196744e6f566) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.018121 27718 raft_consensus.cc:2454] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
W20240503 02:09:18.018277 27524 leader_election.cc:341] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.018317 27524 leader_election.cc:302] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.018594 28507 raft_consensus.cc:2781] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.018630 28654 tablet_replica.cc:330] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.018697 28654 raft_consensus.cc:2229] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.018749 28654 raft_consensus.cc:2258] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.018756 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 58e3e21135574e9bba4f1c6896ee545c (table default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]) successfully deleted
I20240503 02:09:18.018647 28507 raft_consensus.cc:686] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.018857 28507 consensus_queue.cc:237] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.018981 28654 ts_tablet_manager.cc:1875] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.019703 27451 raft_consensus.cc:1261] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.019816 27195 catalog_manager.cc:5484] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.020069 28693 consensus_queue.cc:1035] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.020658 28654 ts_tablet_manager.cc:1888] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.020699 28654 log.cc:1196] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/58740c221e9b422d9fcff96e2df8e85e
I20240503 02:09:18.021180 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:18.020932 28654 ts_tablet_manager.cc:1909] T 58740c221e9b422d9fcff96e2df8e85e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.021534 28654 tablet_replica.cc:330] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.021652 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 58740c221e9b422d9fcff96e2df8e85e (table default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]) successfully deleted
I20240503 02:09:18.021750 28654 raft_consensus.cc:2229] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.021807 28654 raft_consensus.cc:2258] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.021901 28654 ts_tablet_manager.cc:1875] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.022172 28661 ts_tablet_manager.cc:1875] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.023029 28654 ts_tablet_manager.cc:1888] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.023353 28654 log.cc:1196] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3a166dbdc3574a0183e3cc4896307be2
I20240503 02:09:18.023694 28654 ts_tablet_manager.cc:1909] T 3a166dbdc3574a0183e3cc4896307be2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.023742 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 5924b7e1ff81434885296e6e3ed6757b
I20240503 02:09:18.023824 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 5924b7e1ff81434885296e6e3ed6757b deleted and replaced by tablet c934fb82221c45bab114ff8ac97da2bf
I20240503 02:09:18.023907 28654 tablet_replica.cc:330] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.023963 28654 raft_consensus.cc:2229] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.023995 28654 raft_consensus.cc:2258] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.023981 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 5924b7e1ff81434885296e6e3ed6757b with delete_type TABLET_DATA_DELETED (replaced by tablet c934fb82221c45bab114ff8ac97da2bf) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.024035 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3a166dbdc3574a0183e3cc4896307be2 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.024353 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 5924b7e1ff81434885296e6e3ed6757b with delete_type TABLET_DATA_DELETED (replaced by tablet c934fb82221c45bab114ff8ac97da2bf) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.024425 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 5924b7e1ff81434885296e6e3ed6757b with delete_type TABLET_DATA_DELETED (replaced by tablet c934fb82221c45bab114ff8ac97da2bf) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.025619 28654 ts_tablet_manager.cc:1875] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240503 02:09:18.025656 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:18.025795 28661 ts_tablet_manager.cc:1888] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.025838 28661 log.cc:1196] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/58e3e21135574e9bba4f1c6896ee545c
I20240503 02:09:18.026010 28661 ts_tablet_manager.cc:1909] T 58e3e21135574e9bba4f1c6896ee545c P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.026181 28661 tablet_replica.cc:330] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.026258 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 58e3e21135574e9bba4f1c6896ee545c (table default.Table_e7d8d66c8a284fa69d3f8732a45da2be [id=9815edc8c10449dbbce516ae8ca8ac08]) successfully deleted
I20240503 02:09:18.026335 28661 raft_consensus.cc:2229] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.026373 28661 raft_consensus.cc:2258] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.026585 28661 ts_tablet_manager.cc:1875] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.027846 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3a166dbdc3574a0183e3cc4896307be2 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.027927 28653 tablet_replica.cc:330] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.027988 28653 raft_consensus.cc:2229] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.028076 28653 raft_consensus.cc:2258] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.028261 28653 ts_tablet_manager.cc:1875] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.028645 27588 raft_consensus.cc:1261] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.028936 28693 consensus_queue.cc:1035] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.030300 28654 ts_tablet_manager.cc:1888] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.030359 28654 log.cc:1196] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b02d8134f2684fb5a0ddbb9e685c8f79
I20240503 02:09:18.030853 28654 ts_tablet_manager.cc:1909] T b02d8134f2684fb5a0ddbb9e685c8f79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.030977 28653 ts_tablet_manager.cc:1888] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.031033 28653 log.cc:1196] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b02d8134f2684fb5a0ddbb9e685c8f79
I20240503 02:09:18.031186 28654 tablet_replica.cc:330] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.031242 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:18.031283 28654 raft_consensus.cc:2229] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.031306 28653 ts_tablet_manager.cc:1909] T b02d8134f2684fb5a0ddbb9e685c8f79 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.031373 28654 pending_rounds.cc:62] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:18.031477 28654 pending_rounds.cc:68] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420038889099264 op_type: NO_OP noop_request { }
I20240503 02:09:18.031590 28653 tablet_replica.cc:330] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.031678 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:18.031567 28654 raft_consensus.cc:2866] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.031795 28654 raft_consensus.cc:2258] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.031857 28653 raft_consensus.cc:2229] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.031908 28653 raft_consensus.cc:2258] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.032091 28653 ts_tablet_manager.cc:1875] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.032135 28654 ts_tablet_manager.cc:1875] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.033188 28661 ts_tablet_manager.cc:1888] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.033239 28661 log.cc:1196] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/58740c221e9b422d9fcff96e2df8e85e
I20240503 02:09:18.033445 28661 ts_tablet_manager.cc:1909] T 58740c221e9b422d9fcff96e2df8e85e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.033511 28664 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.033553 28664 ts_tablet_manager.cc:1367] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.001s
I20240503 02:09:18.033607 28661 tablet_replica.cc:330] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.033706 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 58740c221e9b422d9fcff96e2df8e85e (table default.Table_98eb120ca20040b89e4cfbdd3e5a75bc [id=582690c0172241e7bbe854e7e31a0b22]) successfully deleted
I20240503 02:09:18.033775 28654 ts_tablet_manager.cc:1888] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.033685 28664 raft_consensus.cc:348] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.033910 28664 raft_consensus.cc:374] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.033926 28664 raft_consensus.cc:729] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.033985 28664 consensus_queue.cc:260] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.034010 28654 log.cc:1196] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fada3a7eedae4a1c9f2372ea5699020f
I20240503 02:09:18.034071 28664 ts_tablet_manager.cc:1398] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.034144 28664 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.034471 28664 tablet_bootstrap.cc:654] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.034808 28410 raft_consensus.cc:482] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.034850 28410 raft_consensus.cc:504] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.034981 28410 leader_election.cc:288] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.034845 28654 ts_tablet_manager.cc:1909] T fada3a7eedae4a1c9f2372ea5699020f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.035380 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd1cb352c40442696afd2fc0bcac0fd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.035408 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd1cb352c40442696afd2fc0bcac0fd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.035454 27718 raft_consensus.cc:2454] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.035475 27448 raft_consensus.cc:2454] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.035604 27524 leader_election.cc:302] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.035718 28410 raft_consensus.cc:2781] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.035741 28410 raft_consensus.cc:482] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.035755 28410 raft_consensus.cc:3037] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.036350 28410 raft_consensus.cc:504] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.036422 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_00d4eb75b1d141c59384c55a3438531e" } modify_external_catalogs: true
I20240503 02:09:18.036456 28410 leader_election.cc:288] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.036475 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_00d4eb75b1d141c59384c55a3438531e" } modify_external_catalogs: true
I20240503 02:09:18.033656 28661 raft_consensus.cc:2229] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.036700 28661 raft_consensus.cc:2258] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.036859 28661 ts_tablet_manager.cc:1875] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.036882 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd1cb352c40442696afd2fc0bcac0fd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.036940 27448 raft_consensus.cc:3037] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.037053 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd1cb352c40442696afd2fc0bcac0fd" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.037109 27718 raft_consensus.cc:3037] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.037561 27448 raft_consensus.cc:2454] T 2fd1cb352c40442696afd2fc0bcac0fd P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.037743 27718 raft_consensus.cc:2454] T 2fd1cb352c40442696afd2fc0bcac0fd P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.037866 27524 leader_election.cc:302] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.037928 28410 raft_consensus.cc:2781] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.037961 28410 raft_consensus.cc:686] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.037993 28410 consensus_queue.cc:237] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.038041 28661 ts_tablet_manager.cc:1888] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.038085 28661 log.cc:1196] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3a166dbdc3574a0183e3cc4896307be2
I20240503 02:09:18.038533 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c9b402f1be4d4fab80a0cef3392725bf
I20240503 02:09:18.039006 28654 tablet_replica.cc:330] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.039108 28654 raft_consensus.cc:2229] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.039186 28654 raft_consensus.cc:2258] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.039309 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fada3a7eedae4a1c9f2372ea5699020f (table default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]) successfully deleted
I20240503 02:09:18.039301 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b402f1be4d4fab80a0cef3392725bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.039494 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b402f1be4d4fab80a0cef3392725bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.039572 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b402f1be4d4fab80a0cef3392725bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.039644 28654 ts_tablet_manager.cc:1875] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.039849 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:18.040181 28661 ts_tablet_manager.cc:1909] T 3a166dbdc3574a0183e3cc4896307be2 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.040455 28661 tablet_replica.cc:330] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.040514 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3a166dbdc3574a0183e3cc4896307be2 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.040787 27198 catalog_manager.cc:5484] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.040863 28654 ts_tablet_manager.cc:1888] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.040903 28654 log.cc:1196] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/5924b7e1ff81434885296e6e3ed6757b
I20240503 02:09:18.041069 28661 raft_consensus.cc:2229] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.041110 28654 ts_tablet_manager.cc:1909] T 5924b7e1ff81434885296e6e3ed6757b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.041157 28661 raft_consensus.cc:2258] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.041287 28653 ts_tablet_manager.cc:1888] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.041375 28661 ts_tablet_manager.cc:1875] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.041460 28653 log.cc:1196] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fada3a7eedae4a1c9f2372ea5699020f
I20240503 02:09:18.041545 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 5924b7e1ff81434885296e6e3ed6757b (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:18.041659 28653 ts_tablet_manager.cc:1909] T fada3a7eedae4a1c9f2372ea5699020f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.041846 28654 tablet_replica.cc:330] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.042049 28654 raft_consensus.cc:2229] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.042070 28653 tablet_replica.cc:330] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.042132 28653 raft_consensus.cc:2229] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.042124 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fada3a7eedae4a1c9f2372ea5699020f (table default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]) successfully deleted
I20240503 02:09:18.042191 28654 raft_consensus.cc:2258] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.042424 29558 tablet.cc:1722] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.042466 28664 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.042505 28664 ts_tablet_manager.cc:1367] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:18.042632 29568 mvcc.cc:205] Tried to move back new op lower bound from 7023420039246422016 to 7023420038959890432. Current Snapshot: MvccSnapshot[applied={T|T < 7023420039246422016}]
I20240503 02:09:18.042642 28664 raft_consensus.cc:348] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.042723 28664 raft_consensus.cc:374] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.042740 28664 raft_consensus.cc:729] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.042763 28661 ts_tablet_manager.cc:1888] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.042776 28664 consensus_queue.cc:260] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.042799 28661 log.cc:1196] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b02d8134f2684fb5a0ddbb9e685c8f79
I20240503 02:09:18.042858 28664 ts_tablet_manager.cc:1398] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.043107 28664 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.043210 28661 ts_tablet_manager.cc:1909] T b02d8134f2684fb5a0ddbb9e685c8f79 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.043474 28664 tablet_bootstrap.cc:654] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.043821 28654 ts_tablet_manager.cc:1875] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.043982 28664 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.044018 28664 ts_tablet_manager.cc:1367] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.044135 28664 raft_consensus.cc:348] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.044188 28664 raft_consensus.cc:374] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.044205 28664 raft_consensus.cc:729] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.044256 28664 consensus_queue.cc:260] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.044335 28664 ts_tablet_manager.cc:1398] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.044409 30009 tablet.cc:1722] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.044991 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:18.046154 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b02d8134f2684fb5a0ddbb9e685c8f79 (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]) successfully deleted
I20240503 02:09:18.046319 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d3802259f32a4bc78ed48977566a8dbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.046428 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3802259f32a4bc78ed48977566a8dbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.047336 28661 tablet_replica.cc:330] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.047420 28661 raft_consensus.cc:2229] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.047456 28661 raft_consensus.cc:2258] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.047644 28664 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.047976 28654 ts_tablet_manager.cc:1888] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.048018 28654 log.cc:1196] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c9b402f1be4d4fab80a0cef3392725bf
I20240503 02:09:18.048084 28664 tablet_bootstrap.cc:654] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.048197 28654 ts_tablet_manager.cc:1909] T c9b402f1be4d4fab80a0cef3392725bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.048589 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c9b402f1be4d4fab80a0cef3392725bf (table default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]) successfully deleted
I20240503 02:09:18.048897 28664 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.048966 28664 ts_tablet_manager.cc:1367] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.049183 28664 raft_consensus.cc:348] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.049268 28664 raft_consensus.cc:374] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.042223 28653 raft_consensus.cc:2258] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.049415 28664 raft_consensus.cc:729] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.049530 28664 consensus_queue.cc:260] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.049690 28664 ts_tablet_manager.cc:1398] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.049746 28661 ts_tablet_manager.cc:1875] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.049768 28664 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.050143 28664 tablet_bootstrap.cc:654] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.049625 28653 ts_tablet_manager.cc:1875] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.050558 28664 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.050601 28664 ts_tablet_manager.cc:1367] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.050714 28664 raft_consensus.cc:348] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.050767 28664 raft_consensus.cc:374] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.050786 28664 raft_consensus.cc:729] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.050814 28664 consensus_queue.cc:260] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.050899 28664 ts_tablet_manager.cc:1398] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.050966 28664 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.051297 28664 tablet_bootstrap.cc:654] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.051993 28664 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.052038 28664 ts_tablet_manager.cc:1367] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.052165 28664 raft_consensus.cc:348] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.052222 28664 raft_consensus.cc:374] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.052246 28664 raft_consensus.cc:729] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.052273 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 0d5f5d7b273241e097bb196744e6f566 (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.052269 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 0d5f5d7b273241e097bb196744e6f566 (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.052309 27692 tablet_service.cc:1467] Processing CreateTablet for tablet a24e57083d8f4dd9b450b0c78a611e0f (DEFAULT_TABLE table=default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.052362 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d5f5d7b273241e097bb196744e6f566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.052362 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d5f5d7b273241e097bb196744e6f566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.052280 28664 consensus_queue.cc:260] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.052429 27560 tablet_service.cc:1467] Processing CreateTablet for tablet c934fb82221c45bab114ff8ac97da2bf (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.052489 28664 ts_tablet_manager.cc:1398] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.052524 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c934fb82221c45bab114ff8ac97da2bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.053380 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24e57083d8f4dd9b450b0c78a611e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.053491 28656 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.053507 27702 tablet_service.cc:1467] Processing CreateTablet for tablet c934fb82221c45bab114ff8ac97da2bf (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.053507 27554 tablet_service.cc:1467] Processing CreateTablet for tablet a24e57083d8f4dd9b450b0c78a611e0f (DEFAULT_TABLE table=default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.053586 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24e57083d8f4dd9b450b0c78a611e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.053599 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c934fb82221c45bab114ff8ac97da2bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.053822 28656 tablet_bootstrap.cc:654] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.054144 28664 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.054463 28664 tablet_bootstrap.cc:654] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.052224 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 633faa7ea971417dbb6116cb7dcb3ef3 (DEFAULT_TABLE table=default.Table_d3802259f32a4bc78ed48977566a8dbd [id=fbcaa9e36eaa424f9a677696bf8d24b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.054695 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 633faa7ea971417dbb6116cb7dcb3ef3 (DEFAULT_TABLE table=default.Table_d3802259f32a4bc78ed48977566a8dbd [id=fbcaa9e36eaa424f9a677696bf8d24b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.054774 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633faa7ea971417dbb6116cb7dcb3ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.054777 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633faa7ea971417dbb6116cb7dcb3ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.055656 28653 ts_tablet_manager.cc:1888] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.055698 28653 log.cc:1196] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/5924b7e1ff81434885296e6e3ed6757b
I20240503 02:09:18.055791 29240 raft_consensus.cc:482] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.055825 29240 raft_consensus.cc:504] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.055877 28653 ts_tablet_manager.cc:1909] T 5924b7e1ff81434885296e6e3ed6757b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.055932 29240 leader_election.cc:288] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.056421 27425 tablet_service.cc:1467] Processing CreateTablet for tablet c934fb82221c45bab114ff8ac97da2bf (DEFAULT_TABLE table=default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.056515 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c934fb82221c45bab114ff8ac97da2bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.056919 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647ba8fd2b324e8884e22e16ca414471" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.056977 27720 raft_consensus.cc:2454] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.057446 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 0d5f5d7b273241e097bb196744e6f566 (DEFAULT_TABLE table=default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.057541 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d5f5d7b273241e097bb196744e6f566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.057696 27523 leader_election.cc:302] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.057732 27422 tablet_service.cc:1467] Processing CreateTablet for tablet a24e57083d8f4dd9b450b0c78a611e0f (DEFAULT_TABLE table=default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.057785 29240 raft_consensus.cc:2781] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.057801 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24e57083d8f4dd9b450b0c78a611e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.057816 29240 raft_consensus.cc:482] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.057833 29240 raft_consensus.cc:3037] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.058382 29240 raft_consensus.cc:504] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.058490 29240 leader_election.cc:288] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.058616 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647ba8fd2b324e8884e22e16ca414471" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.058673 27720 raft_consensus.cc:3037] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.058693 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647ba8fd2b324e8884e22e16ca414471" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.058754 27447 raft_consensus.cc:3037] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.058863 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647ba8fd2b324e8884e22e16ca414471" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.059279 29240 raft_consensus.cc:482] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.059305 27720 raft_consensus.cc:2454] T 647ba8fd2b324e8884e22e16ca414471 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.059324 27447 raft_consensus.cc:2454] T 647ba8fd2b324e8884e22e16ca414471 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.059326 29240 raft_consensus.cc:504] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.059417 29240 leader_election.cc:288] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.059406 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 633faa7ea971417dbb6116cb7dcb3ef3 (DEFAULT_TABLE table=default.Table_d3802259f32a4bc78ed48977566a8dbd [id=fbcaa9e36eaa424f9a677696bf8d24b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.059486 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633faa7ea971417dbb6116cb7dcb3ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.059473 27524 leader_election.cc:302] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.059548 29240 raft_consensus.cc:2781] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.059584 29240 raft_consensus.cc:686] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.059561 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.059616 29240 consensus_queue.cc:237] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:18.059708 27524 leader_election.cc:341] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.060144 28656 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.060179 28656 ts_tablet_manager.cc:1367] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.060274 28656 raft_consensus.cc:348] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.060307 28656 raft_consensus.cc:374] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.060317 28656 raft_consensus.cc:729] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.060333 28656 consensus_queue.cc:260] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.060405 28656 ts_tablet_manager.cc:1398] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.060449 28656 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.060746 28656 tablet_bootstrap.cc:654] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.060859 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7c554f7ea6f74725a303fabd9ae42cda" } modify_external_catalogs: true
I20240503 02:09:18.060921 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_7c554f7ea6f74725a303fabd9ae42cda" } modify_external_catalogs: true
I20240503 02:09:18.061158 27588 raft_consensus.cc:1261] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.061270 27447 raft_consensus.cc:1261] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.061523 27853 consensus_queue.cc:1035] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.061620 29643 consensus_queue.cc:1035] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.061671 28661 ts_tablet_manager.cc:1888] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.061713 28661 log.cc:1196] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fada3a7eedae4a1c9f2372ea5699020f
I20240503 02:09:18.061849 28661 ts_tablet_manager.cc:1909] T fada3a7eedae4a1c9f2372ea5699020f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.062471 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 7a559ab04d7c4b148c054656cc3863db
I20240503 02:09:18.062494 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.062548 27720 raft_consensus.cc:2454] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.062691 27523 leader_election.cc:302] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.062709 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a559ab04d7c4b148c054656cc3863db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.062690 28410 raft_consensus.cc:482] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.062837 28410 raft_consensus.cc:504] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.062919 28410 leader_election.cc:288] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.062949 28410 raft_consensus.cc:2781] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.062965 28410 raft_consensus.cc:482] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.062980 28410 raft_consensus.cc:3037] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.063140 27853 raft_consensus.cc:482] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.063158 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a23e797aee94bcb90cbb7545147a893" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.063206 27448 raft_consensus.cc:2454] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.063184 27853 raft_consensus.cc:504] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.063270 27853 leader_election.cc:288] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.063292 28654 tablet_replica.cc:330] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.063352 28654 raft_consensus.cc:2229] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.063382 28654 raft_consensus.cc:2258] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.063390 27524 leader_election.cc:302] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.063484 28507 raft_consensus.cc:2781] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.063519 28507 raft_consensus.cc:482] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.063537 28507 raft_consensus.cc:3037] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.063558 28410 raft_consensus.cc:504] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.063640 28410 leader_election.cc:288] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.063634 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3979ea6b19f443d1a96deefb8acb7867 (table default.Table_b1dcd1438e66456ea9ed0228abb696da [id=218fc00f3dd74e29877481adf6114a18]). Expected version 1 got 0
I20240503 02:09:18.063691 29558 tablet.cc:1722] T 3979ea6b19f443d1a96deefb8acb7867 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.063910 28654 ts_tablet_manager.cc:1875] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.064060 28656 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.064092 28656 ts_tablet_manager.cc:1367] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:18.064128 28507 raft_consensus.cc:504] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.064232 28507 leader_election.cc:288] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.064289 28656 raft_consensus.cc:348] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.064339 28656 raft_consensus.cc:374] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.064356 28656 raft_consensus.cc:729] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.064379 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a23e797aee94bcb90cbb7545147a893" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.064393 28656 consensus_queue.cc:260] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.064432 27719 raft_consensus.cc:3037] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.064471 28656 ts_tablet_manager.cc:1398] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.064519 28656 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.064831 28656 tablet_bootstrap.cc:654] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.065081 27719 raft_consensus.cc:2454] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.065086 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.065167 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a559ab04d7c4b148c054656cc3863db" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.065201 27718 raft_consensus.cc:3037] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.065222 27452 raft_consensus.cc:2454] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.065284 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a23e797aee94bcb90cbb7545147a893" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.065367 27453 raft_consensus.cc:3037] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.065368 27659 leader_election.cc:302] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.065596 28664 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.065634 28664 ts_tablet_manager.cc:1367] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:18.065745 28664 raft_consensus.cc:348] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.065802 28664 raft_consensus.cc:374] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.065814 28693 raft_consensus.cc:2781] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.065841 28664 raft_consensus.cc:729] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.065778 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a23e797aee94bcb90cbb7545147a893" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.065847 28693 raft_consensus.cc:482] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.065932 28693 raft_consensus.cc:3037] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.065968 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a559ab04d7c4b148c054656cc3863db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.066012 27453 raft_consensus.cc:2454] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.066138 27524 leader_election.cc:302] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.066202 28507 raft_consensus.cc:2781] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.066231 28507 raft_consensus.cc:686] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.066260 28507 consensus_queue.cc:237] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.066269 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.066551 28693 raft_consensus.cc:504] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.066577 27198 catalog_manager.cc:5484] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.066649 28693 leader_election.cc:288] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.066965 28654 ts_tablet_manager.cc:1888] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.067008 28654 log.cc:1196] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/7a559ab04d7c4b148c054656cc3863db
I20240503 02:09:18.067145 28654 ts_tablet_manager.cc:1909] T 7a559ab04d7c4b148c054656cc3863db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
W20240503 02:09:18.067165 27524 leader_election.cc:341] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.065794 27718 raft_consensus.cc:2454] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.065882 28664 consensus_queue.cc:260] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.067420 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 7a559ab04d7c4b148c054656cc3863db (table default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]) successfully deleted
W20240503 02:09:18.067441 27656 leader_election.cc:341] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 7a559ab04d7c4b148c054656cc3863db
W20240503 02:09:18.067492 27656 leader_election.cc:341] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Not found: Tablet not found: 7a559ab04d7c4b148c054656cc3863db
I20240503 02:09:18.067503 28664 ts_tablet_manager.cc:1398] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.067576 28664 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.067647 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a559ab04d7c4b148c054656cc3863db" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.067701 27448 raft_consensus.cc:3037] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.067819 30190 tablet.cc:1722] T 3979ea6b19f443d1a96deefb8acb7867 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.067914 28664 tablet_bootstrap.cc:654] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.065876 27720 raft_consensus.cc:2362] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:18.068301 27448 raft_consensus.cc:2454] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.068521 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8121b25900c6441c9ccd1c09f0a94276"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.068562 28653 tablet_replica.cc:330] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.068624 28653 raft_consensus.cc:2229] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
W20240503 02:09:18.068624 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8121b25900c6441c9ccd1c09f0a94276 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.068676 28653 raft_consensus.cc:2258] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.068819 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 5924b7e1ff81434885296e6e3ed6757b (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:18.069195 27523 leader_election.cc:302] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.069404 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a559ab04d7c4b148c054656cc3863db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.069494 29240 raft_consensus.cc:2781] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.069540 29240 raft_consensus.cc:686] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.069572 29240 consensus_queue.cc:237] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.069808 28655 tablet_bootstrap.cc:492] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.069849 28655 ts_tablet_manager.cc:1367] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.084s user 0.000s sys 0.001s
I20240503 02:09:18.069867 27659 leader_election.cc:302] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:18.069947 28740 raft_consensus.cc:2781] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.069985 28740 raft_consensus.cc:686] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.069989 28655 raft_consensus.cc:348] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.070035 28655 raft_consensus.cc:374] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.070021 28740 consensus_queue.cc:237] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.070052 28655 raft_consensus.cc:729] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.070087 28655 consensus_queue.cc:260] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.070176 28655 ts_tablet_manager.cc:1398] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.070235 28655 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.070554 28655 tablet_bootstrap.cc:654] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.072660 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fada3a7eedae4a1c9f2372ea5699020f (table default.Table_cf18641e208546c391738b4e80b51bd7 [id=0616202bc930476a9c2c064abadbf87e]) successfully deleted
I20240503 02:09:18.072688 28661 tablet_replica.cc:330] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.072959 28774 raft_consensus.cc:482] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.073006 28774 raft_consensus.cc:504] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.073065 27703 tablet_service.cc:1467] Processing CreateTablet for tablet dcc8f22f822e4b3e829bba41bd702f0c (DEFAULT_TABLE table=default.Table_8121b25900c6441c9ccd1c09f0a94276 [id=628e12a3fb71416f8243647838652fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.073093 28774 leader_election.cc:288] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.073153 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc8f22f822e4b3e829bba41bd702f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.073491 27560 tablet_service.cc:1467] Processing CreateTablet for tablet dcc8f22f822e4b3e829bba41bd702f0c (DEFAULT_TABLE table=default.Table_8121b25900c6441c9ccd1c09f0a94276 [id=628e12a3fb71416f8243647838652fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.073544 27426 tablet_service.cc:1467] Processing CreateTablet for tablet dcc8f22f822e4b3e829bba41bd702f0c (DEFAULT_TABLE table=default.Table_8121b25900c6441c9ccd1c09f0a94276 [id=628e12a3fb71416f8243647838652fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.073581 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc8f22f822e4b3e829bba41bd702f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.073629 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc8f22f822e4b3e829bba41bd702f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.074262 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.074319 27582 raft_consensus.cc:2454] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.074442 27656 leader_election.cc:302] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.074508 28774 raft_consensus.cc:2781] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.074530 28774 raft_consensus.cc:482] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.074546 28774 raft_consensus.cc:3037] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.074640 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.074944 28661 raft_consensus.cc:2229] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
W20240503 02:09:18.075001 27659 leader_election.cc:341] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.075099 28653 ts_tablet_manager.cc:1875] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.075125 28774 raft_consensus.cc:504] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.075241 28774 leader_election.cc:288] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.075011 28661 raft_consensus.cc:2258] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.075428 29551 raft_consensus.cc:482] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.075466 29551 raft_consensus.cc:504] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.075497 28661 ts_tablet_manager.cc:1875] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.075544 29551 leader_election.cc:288] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.075594 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.075665 27586 raft_consensus.cc:3037] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.076184 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.076256 27720 raft_consensus.cc:2379] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:18.076263 27586 raft_consensus.cc:2454] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.076462 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_bd478bdb5a454d489940b1dddf851c66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.076581 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd478bdb5a454d489940b1dddf851c66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.076967 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.077070 28710 tablet.cc:1722] T 3979ea6b19f443d1a96deefb8acb7867 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.077164 27656 leader_election.cc:302] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.077227 27853 raft_consensus.cc:2781] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.077258 27853 raft_consensus.cc:686] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.077287 27853 consensus_queue.cc:237] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
W20240503 02:09:18.078469 27659 leader_election.cc:341] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.079188 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.079239 27198 catalog_manager.cc:5484] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.079324 27198 catalog_manager.cc:5484] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.079384 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea94da2559e423a9dbb0cb8bc50cb8b" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.079589 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b363bfa5d6f04d658cec669b62811c4f (DEFAULT_TABLE table=default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.079680 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b363bfa5d6f04d658cec669b62811c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.079707 27386 leader_election.cc:302] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.079802 29550 raft_consensus.cc:3037] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.080197 27560 tablet_service.cc:1467] Processing CreateTablet for tablet b363bfa5d6f04d658cec669b62811c4f (DEFAULT_TABLE table=default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.080277 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b363bfa5d6f04d658cec669b62811c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.080376 29550 raft_consensus.cc:2726] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.081313 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b363bfa5d6f04d658cec669b62811c4f (DEFAULT_TABLE table=default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.081425 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b363bfa5d6f04d658cec669b62811c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.081532 28664 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.081569 28664 ts_tablet_manager.cc:1367] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
W20240503 02:09:18.081650 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet a24e57083d8f4dd9b450b0c78a611e0f consensus state has no leader
I20240503 02:09:18.081692 28664 raft_consensus.cc:348] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.081743 28664 raft_consensus.cc:374] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.081760 28664 raft_consensus.cc:729] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.081796 28664 consensus_queue.cc:260] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.081871 28664 ts_tablet_manager.cc:1398] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.081914 28664 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.082274 28664 tablet_bootstrap.cc:654] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.082628 27195 catalog_manager.cc:5484] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.082736 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.084461 28656 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.084507 28656 ts_tablet_manager.cc:1367] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240503 02:09:18.084640 28656 raft_consensus.cc:348] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.084692 28656 raft_consensus.cc:374] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.084708 28656 raft_consensus.cc:729] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.084764 28656 consensus_queue.cc:260] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.084883 28656 ts_tablet_manager.cc:1398] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.085904 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]). Expected version 2 got 0
I20240503 02:09:18.086462 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.086530 28661 ts_tablet_manager.cc:1888] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.086575 28661 log.cc:1196] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/5924b7e1ff81434885296e6e3ed6757b
I20240503 02:09:18.086736 28656 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.086939 28661 ts_tablet_manager.cc:1909] T 5924b7e1ff81434885296e6e3ed6757b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.087095 28656 tablet_bootstrap.cc:654] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.087211 28661 tablet_replica.cc:330] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.087271 28661 raft_consensus.cc:2229] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.087260 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 5924b7e1ff81434885296e6e3ed6757b (table default.Table_2949eee1a0ab47ca85d18131e12e8ab8 [id=c132998c9cdb4321b7a24952a72e6a9f]) successfully deleted
I20240503 02:09:18.087311 28661 raft_consensus.cc:2258] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.087574 28661 ts_tablet_manager.cc:1875] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.087777 28710 tablet.cc:1722] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.089008 28653 ts_tablet_manager.cc:1888] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.089116 28653 log.cc:1196] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c9b402f1be4d4fab80a0cef3392725bf
I20240503 02:09:18.089301 28661 ts_tablet_manager.cc:1888] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.089422 28661 log.cc:1196] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c9b402f1be4d4fab80a0cef3392725bf
I20240503 02:09:18.089407 28653 ts_tablet_manager.cc:1909] T c9b402f1be4d4fab80a0cef3392725bf P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.089556 28656 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.089602 28656 ts_tablet_manager.cc:1367] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.089742 28661 ts_tablet_manager.cc:1909] T c9b402f1be4d4fab80a0cef3392725bf P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.089744 28656 raft_consensus.cc:348] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.089800 28656 raft_consensus.cc:374] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.089821 28656 raft_consensus.cc:729] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.089857 28656 consensus_queue.cc:260] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.089982 28656 ts_tablet_manager.cc:1398] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.090058 28656 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.090118 28653 tablet_replica.cc:330] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.090179 28653 raft_consensus.cc:2229] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.090205 28653 raft_consensus.cc:2258] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.090385 28653 ts_tablet_manager.cc:1875] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.090389 28656 tablet_bootstrap.cc:654] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.090555 28661 tablet_replica.cc:330] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.090582 28655 tablet_bootstrap.cc:492] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.090626 28655 ts_tablet_manager.cc:1367] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20240503 02:09:18.090711 28661 raft_consensus.cc:2229] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.090764 28655 raft_consensus.cc:348] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.090802 28661 pending_rounds.cc:62] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:18.090832 28655 raft_consensus.cc:374] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.090858 28655 raft_consensus.cc:729] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.090885 28655 consensus_queue.cc:260] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.090922 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c9b402f1be4d4fab80a0cef3392725bf (table default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]) successfully deleted
I20240503 02:09:18.090982 28655 ts_tablet_manager.cc:1398] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.091044 28655 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.090919 28661 pending_rounds.cc:68] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420039454973952 op_type: NO_OP noop_request { }
I20240503 02:09:18.091194 28661 raft_consensus.cc:2866] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.091264 28661 raft_consensus.cc:2258] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.091375 28655 tablet_bootstrap.cc:654] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.091600 28661 ts_tablet_manager.cc:1875] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.092245 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c9b402f1be4d4fab80a0cef3392725bf (table default.Table_00d4eb75b1d141c59384c55a3438531e [id=4061165baa3f45e49cb3117b2c6f9119]) successfully deleted
I20240503 02:09:18.093370 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_6baf0a2db8924e6dafddd9699daf92a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.093484 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6baf0a2db8924e6dafddd9699daf92a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.093885 28661 ts_tablet_manager.cc:1888] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.094023 28661 log.cc:1196] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/7a559ab04d7c4b148c054656cc3863db
I20240503 02:09:18.094161 28655 tablet_bootstrap.cc:492] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.094208 28655 ts_tablet_manager.cc:1367] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:18.094359 28661 ts_tablet_manager.cc:1909] T 7a559ab04d7c4b148c054656cc3863db P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.094343 28655 raft_consensus.cc:348] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.094422 28655 raft_consensus.cc:374] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.094441 28655 raft_consensus.cc:729] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.094478 28655 consensus_queue.cc:260] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.094560 28655 ts_tablet_manager.cc:1398] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.094624 28655 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.094686 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 7a559ab04d7c4b148c054656cc3863db (table default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]) successfully deleted
I20240503 02:09:18.094851 28656 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.094890 28656 ts_tablet_manager.cc:1367] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.095016 28656 raft_consensus.cc:348] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.095050 28655 tablet_bootstrap.cc:654] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.095078 28656 raft_consensus.cc:374] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.095098 28656 raft_consensus.cc:729] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.095144 28656 consensus_queue.cc:260] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.095247 28656 ts_tablet_manager.cc:1398] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.095602 28656 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.096187 28656 tablet_bootstrap.cc:654] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.096694 28656 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.096735 28656 ts_tablet_manager.cc:1367] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.096875 28656 raft_consensus.cc:348] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.096868 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_fdd5c3c3cfca4f37a9458a54d72fc7f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.096947 28656 raft_consensus.cc:374] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.096966 28656 raft_consensus.cc:729] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
W20240503 02:09:18.096971 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdd5c3c3cfca4f37a9458a54d72fc7f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.097005 28656 consensus_queue.cc:260] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.097096 28656 ts_tablet_manager.cc:1398] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.097354 27853 raft_consensus.cc:482] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.097404 27853 raft_consensus.cc:504] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.097501 27853 leader_election.cc:288] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.097649 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b168c17ac44030a73596e9d79123a7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.097721 27451 raft_consensus.cc:2454] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.097942 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b168c17ac44030a73596e9d79123a7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.098004 27586 raft_consensus.cc:2454] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.098100 27656 leader_election.cc:302] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.098199 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7ccfcbdc854a09b17f6b03897ec63f (DEFAULT_TABLE table=default.Table_6baf0a2db8924e6dafddd9699daf92a3 [id=697489e95c56421a81fd2de521846961]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.098242 27853 raft_consensus.cc:2781] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.098271 27853 raft_consensus.cc:482] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.098289 27853 raft_consensus.cc:3037] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.098291 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7ccfcbdc854a09b17f6b03897ec63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.098913 27853 raft_consensus.cc:504] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.099021 27853 leader_election.cc:288] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.099167 28655 tablet_bootstrap.cc:492] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.099205 28655 ts_tablet_manager.cc:1367] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.099243 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_11afbe28398844b19ac93c4f5d6971b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.099336 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11afbe28398844b19ac93c4f5d6971b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.099328 28655 raft_consensus.cc:348] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.099380 28655 raft_consensus.cc:374] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.099409 28655 raft_consensus.cc:729] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.099448 28655 consensus_queue.cc:260] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.099534 28655 ts_tablet_manager.cc:1398] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.099550 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b168c17ac44030a73596e9d79123a7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.099617 27451 raft_consensus.cc:3037] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.099627 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b168c17ac44030a73596e9d79123a7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.099671 27586 raft_consensus.cc:3037] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.100030 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7ccfcbdc854a09b17f6b03897ec63f (DEFAULT_TABLE table=default.Table_6baf0a2db8924e6dafddd9699daf92a3 [id=697489e95c56421a81fd2de521846961]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.100126 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7ccfcbdc854a09b17f6b03897ec63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.100205 27586 raft_consensus.cc:2454] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.100203 27451 raft_consensus.cc:2454] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.099583 28655 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.100368 27659 leader_election.cc:302] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.100653 28655 tablet_bootstrap.cc:654] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.100888 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_5464160fc93a4ad7ae9ad9ad8ab45a2e" } new_table_name: "default.Table_9c9ffc9850cd4336a8d59377649a18e6" modify_external_catalogs: true
I20240503 02:09:18.101141 28656 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.101202 28655 tablet_bootstrap.cc:492] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.101235 28655 ts_tablet_manager.cc:1367] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.101344 27702 tablet_service.cc:1467] Processing CreateTablet for tablet e76ee4b127b944ee8611ac188dbcc087 (DEFAULT_TABLE table=default.Table_fdd5c3c3cfca4f37a9458a54d72fc7f1 [id=4cc712a3acb642248bd8a669ad5fe23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.101380 28655 raft_consensus.cc:348] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.101436 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76ee4b127b944ee8611ac188dbcc087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.101454 28655 raft_consensus.cc:374] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.101475 28655 raft_consensus.cc:729] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.101498 28656 tablet_bootstrap.cc:654] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.101511 28655 consensus_queue.cc:260] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.101596 28655 ts_tablet_manager.cc:1398] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.101644 28655 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.101954 28655 tablet_bootstrap.cc:654] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.102015 27560 tablet_service.cc:1467] Processing CreateTablet for tablet e76ee4b127b944ee8611ac188dbcc087 (DEFAULT_TABLE table=default.Table_fdd5c3c3cfca4f37a9458a54d72fc7f1 [id=4cc712a3acb642248bd8a669ad5fe23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.102100 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76ee4b127b944ee8611ac188dbcc087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.102274 27853 raft_consensus.cc:2781] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.102321 27853 raft_consensus.cc:686] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.102356 27853 consensus_queue.cc:237] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.102568 28655 tablet_bootstrap.cc:492] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.102602 28655 ts_tablet_manager.cc:1367] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.102721 28655 raft_consensus.cc:348] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.102767 28655 raft_consensus.cc:374] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.102782 28655 raft_consensus.cc:729] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.102813 28655 consensus_queue.cc:260] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.102882 28655 ts_tablet_manager.cc:1398] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.102926 28655 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.102918 28693 raft_consensus.cc:482] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.102963 28693 raft_consensus.cc:504] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.103049 28693 leader_election.cc:288] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.103147 27422 tablet_service.cc:1467] Processing CreateTablet for tablet e76ee4b127b944ee8611ac188dbcc087 (DEFAULT_TABLE table=default.Table_fdd5c3c3cfca4f37a9458a54d72fc7f1 [id=4cc712a3acb642248bd8a669ad5fe23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.103165 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 6f7ccfcbdc854a09b17f6b03897ec63f (DEFAULT_TABLE table=default.Table_6baf0a2db8924e6dafddd9699daf92a3 [id=697489e95c56421a81fd2de521846961]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.103235 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f7ccfcbdc854a09b17f6b03897ec63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.103253 28655 tablet_bootstrap.cc:654] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.103449 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e76ee4b127b944ee8611ac188dbcc087. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.104202 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59aa5f0d43d14cf8bdafc8bb7fedb0f7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.104256 27451 raft_consensus.cc:2454] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.104387 27659 leader_election.cc:302] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.104455 28693 raft_consensus.cc:2781] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.104478 28693 raft_consensus.cc:482] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.104494 28693 raft_consensus.cc:3037] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.104951 28656 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.104993 28656 ts_tablet_manager.cc:1367] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:18.105094 28693 raft_consensus.cc:504] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.105106 28656 raft_consensus.cc:348] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.105170 28656 raft_consensus.cc:374] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.105190 28656 raft_consensus.cc:729] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.105190 28693 leader_election.cc:288] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.105221 28656 consensus_queue.cc:260] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.105298 28656 ts_tablet_manager.cc:1398] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.105365 28656 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.105489 28664 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.105525 28664 ts_tablet_manager.cc:1367] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.000s
I20240503 02:09:18.105651 28664 raft_consensus.cc:348] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.105700 28664 raft_consensus.cc:374] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.105716 28664 raft_consensus.cc:729] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.105715 28656 tablet_bootstrap.cc:654] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.105754 28664 consensus_queue.cc:260] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.105868 28664 ts_tablet_manager.cc:1398] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.105921 28664 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.106143 27198 catalog_manager.cc:5484] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.106246 28664 tablet_bootstrap.cc:654] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.105280 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 6c869e9a2d9a4472a65d57af51b491a1 (DEFAULT_TABLE table=default.Table_11afbe28398844b19ac93c4f5d6971b6 [id=57a07f9e33fb42ba9c221534555eab31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.106326 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c869e9a2d9a4472a65d57af51b491a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.106890 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 6c869e9a2d9a4472a65d57af51b491a1 (DEFAULT_TABLE table=default.Table_11afbe28398844b19ac93c4f5d6971b6 [id=57a07f9e33fb42ba9c221534555eab31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.106971 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c869e9a2d9a4472a65d57af51b491a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.108110 28655 tablet_bootstrap.cc:492] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.108145 28655 ts_tablet_manager.cc:1367] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.108289 28655 raft_consensus.cc:348] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.108341 28655 raft_consensus.cc:374] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.108358 28655 raft_consensus.cc:729] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.108386 28655 consensus_queue.cc:260] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.108470 28655 ts_tablet_manager.cc:1398] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.108520 28655 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.108551 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6c869e9a2d9a4472a65d57af51b491a1 (DEFAULT_TABLE table=default.Table_11afbe28398844b19ac93c4f5d6971b6 [id=57a07f9e33fb42ba9c221534555eab31]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.108636 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c869e9a2d9a4472a65d57af51b491a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.108834 28655 tablet_bootstrap.cc:654] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.109086 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59aa5f0d43d14cf8bdafc8bb7fedb0f7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.109143 27586 raft_consensus.cc:2454] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.109217 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59aa5f0d43d14cf8bdafc8bb7fedb0f7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.109261 27588 raft_consensus.cc:3037] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.109875 27588 raft_consensus.cc:2454] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.109952 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59aa5f0d43d14cf8bdafc8bb7fedb0f7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.109998 27451 raft_consensus.cc:3037] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.110586 27451 raft_consensus.cc:2454] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.110806 27659 leader_election.cc:302] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.110879 28693 raft_consensus.cc:2781] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.110883 28653 ts_tablet_manager.cc:1888] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.110925 28693 raft_consensus.cc:686] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.110958 28693 consensus_queue.cc:237] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.110929 28653 log.cc:1196] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/7a559ab04d7c4b148c054656cc3863db
I20240503 02:09:18.110981 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8b859087b6e64ad6ac2a7d426f69e6d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.111069 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b859087b6e64ad6ac2a7d426f69e6d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.111174 28653 ts_tablet_manager.cc:1909] T 7a559ab04d7c4b148c054656cc3863db P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.112439 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5128eedd382c4e07a6f1b4933a8800e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.112465 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_eab18f79ec204528b7d1a22f3cbdc354"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.112529 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5128eedd382c4e07a6f1b4933a8800e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20240503 02:09:18.112560 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eab18f79ec204528b7d1a22f3cbdc354 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.113250 27198 catalog_manager.cc:5484] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.113737 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 7a559ab04d7c4b148c054656cc3863db (table default.Table_7c554f7ea6f74725a303fabd9ae42cda [id=3e78472b700043129cda646e8ec5e607]) successfully deleted
I20240503 02:09:18.114672 28655 tablet_bootstrap.cc:492] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.114725 28655 ts_tablet_manager.cc:1367] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.114872 28710 tablet.cc:1722] T 63e82b82ee7649d88867dbb9eef8a9de P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.115068 30190 tablet.cc:1722] T 63e82b82ee7649d88867dbb9eef8a9de P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.114848 28655 raft_consensus.cc:348] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.115164 28655 raft_consensus.cc:374] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.115190 28655 raft_consensus.cc:729] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.115227 28655 consensus_queue.cc:260] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.115314 28655 ts_tablet_manager.cc:1398] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.115381 28655 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.115720 28655 tablet_bootstrap.cc:654] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.115768 28664 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.115778 28656 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.115806 28664 ts_tablet_manager.cc:1367] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.000s
I20240503 02:09:18.115814 28656 ts_tablet_manager.cc:1367] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:18.115909 28656 raft_consensus.cc:348] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.115952 28656 raft_consensus.cc:374] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.115965 28656 raft_consensus.cc:729] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.115942 28664 raft_consensus.cc:348] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.115993 28664 raft_consensus.cc:374] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.116009 28664 raft_consensus.cc:729] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.116000 28656 consensus_queue.cc:260] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.116043 28664 consensus_queue.cc:260] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.116080 28656 ts_tablet_manager.cc:1398] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.116123 28664 ts_tablet_manager.cc:1398] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.116147 28656 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.116190 28655 tablet_bootstrap.cc:492] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.116202 28664 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.116221 28655 ts_tablet_manager.cc:1367] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.116346 28655 raft_consensus.cc:348] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.116389 28655 raft_consensus.cc:374] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.116405 28655 raft_consensus.cc:729] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.116441 28655 consensus_queue.cc:260] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.116472 28656 tablet_bootstrap.cc:654] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.116521 28655 ts_tablet_manager.cc:1398] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.116542 28664 tablet_bootstrap.cc:654] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.116578 28655 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.116890 28655 tablet_bootstrap.cc:654] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.118799 27702 tablet_service.cc:1467] Processing CreateTablet for tablet a551225a4c8f4e4f84410bcb6fa38634 (DEFAULT_TABLE table=default.Table_8b859087b6e64ad6ac2a7d426f69e6d1 [id=e9d7bbda984d4b358f204abf5ac862f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.118898 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a551225a4c8f4e4f84410bcb6fa38634. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.119167 28656 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.119204 28656 ts_tablet_manager.cc:1367] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.119302 28656 raft_consensus.cc:348] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.119352 28656 raft_consensus.cc:374] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.119371 28656 raft_consensus.cc:729] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.119405 28656 consensus_queue.cc:260] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.119484 28656 ts_tablet_manager.cc:1398] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.119797 28655 tablet_bootstrap.cc:492] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.119834 28655 ts_tablet_manager.cc:1367] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:18.119915 27422 tablet_service.cc:1467] Processing CreateTablet for tablet a551225a4c8f4e4f84410bcb6fa38634 (DEFAULT_TABLE table=default.Table_8b859087b6e64ad6ac2a7d426f69e6d1 [id=e9d7bbda984d4b358f204abf5ac862f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.119966 28655 raft_consensus.cc:348] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.120029 28655 raft_consensus.cc:374] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.120050 28655 raft_consensus.cc:729] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.120081 28655 consensus_queue.cc:260] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.120175 28655 ts_tablet_manager.cc:1398] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.120195 27560 tablet_service.cc:1467] Processing CreateTablet for tablet a551225a4c8f4e4f84410bcb6fa38634 (DEFAULT_TABLE table=default.Table_8b859087b6e64ad6ac2a7d426f69e6d1 [id=e9d7bbda984d4b358f204abf5ac862f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.120227 28655 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.120271 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a551225a4c8f4e4f84410bcb6fa38634. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.120540 29558 tablet.cc:1722] T 63e82b82ee7649d88867dbb9eef8a9de P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.119999 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a551225a4c8f4e4f84410bcb6fa38634. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.121737 28655 tablet_bootstrap.cc:654] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.121994 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3e4e78e5e04446d2b2287cf636ec1e5f" } modify_external_catalogs: true
I20240503 02:09:18.122006 27703 tablet_service.cc:1467] Processing CreateTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.122045 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3e4e78e5e04446d2b2287cf636ec1e5f" } modify_external_catalogs: true
I20240503 02:09:18.122090 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb56205ff0d4a3ba73d21d9e821a43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.122300 28664 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.122334 28664 ts_tablet_manager.cc:1367] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:18.122478 28664 raft_consensus.cc:348] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.122524 28664 raft_consensus.cc:374] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.122541 28664 raft_consensus.cc:729] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.122577 28664 consensus_queue.cc:260] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.122628 27565 tablet_service.cc:1467] Processing CreateTablet for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8 (DEFAULT_TABLE table=default.Table_5128eedd382c4e07a6f1b4933a8800e3 [id=902feb7df4de456ebd4415c6781396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.122670 28664 ts_tablet_manager.cc:1398] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.122715 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.122732 28664 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.123066 28664 tablet_bootstrap.cc:654] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.123087 28655 tablet_bootstrap.cc:492] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.123122 28655 ts_tablet_manager.cc:1367] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.123237 28655 raft_consensus.cc:348] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.123294 28655 raft_consensus.cc:374] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.123312 28655 raft_consensus.cc:729] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.123345 28655 consensus_queue.cc:260] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.123347 27554 tablet_service.cc:1467] Processing CreateTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.123448 28655 ts_tablet_manager.cc:1398] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.123463 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb56205ff0d4a3ba73d21d9e821a43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.123502 28655 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.123836 28655 tablet_bootstrap.cc:654] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.123844 27702 tablet_service.cc:1467] Processing CreateTablet for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8 (DEFAULT_TABLE table=default.Table_5128eedd382c4e07a6f1b4933a8800e3 [id=902feb7df4de456ebd4415c6781396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.123936 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.124287 28656 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.124289 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9c9ffc9850cd4336a8d59377649a18e6 [id=4908b7d914c64d328883e411ea95a2b5] alter complete (version 1)
I20240503 02:09:18.124514 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_d6c7af51340244c2b242eab1f705a0d8" } new_table_name: "default.Table_3cbd9d3dfc744f60a112172d69ea0cc0" modify_external_catalogs: true
I20240503 02:09:18.124635 28656 tablet_bootstrap.cc:654] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.125146 27422 tablet_service.cc:1467] Processing CreateTablet for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8 (DEFAULT_TABLE table=default.Table_5128eedd382c4e07a6f1b4933a8800e3 [id=902feb7df4de456ebd4415c6781396a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.125232 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e60cbc3e764f71bcfb0ae5d0558fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.125746 27426 tablet_service.cc:1467] Processing CreateTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.125825 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acb56205ff0d4a3ba73d21d9e821a43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.126952 29550 consensus_queue.cc:1035] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.127297 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f2f62c59d2e24782ba20fb8d4f8bec03
I20240503 02:09:18.127734 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.127877 28661 tablet_replica.cc:330] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.127966 28661 raft_consensus.cc:2229] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.128039 28661 raft_consensus.cc:2258] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.128100 28664 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.128137 28664 ts_tablet_manager.cc:1367] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.128254 28664 raft_consensus.cc:348] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.127892 30097 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:18.128298 28664 raft_consensus.cc:374] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.128314 28664 raft_consensus.cc:729] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.128353 28664 consensus_queue.cc:260] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.128429 28664 ts_tablet_manager.cc:1398] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.128556 28664 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.128876 28664 tablet_bootstrap.cc:654] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.128911 30190 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:18.129536 28661 ts_tablet_manager.cc:1875] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.129557 28656 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.129565 28655 tablet_bootstrap.cc:492] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.129590 28656 ts_tablet_manager.cc:1367] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.129601 28655 ts_tablet_manager.cc:1367] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.129717 28656 raft_consensus.cc:348] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.129727 28655 raft_consensus.cc:348] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.129762 28656 raft_consensus.cc:374] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.129776 28655 raft_consensus.cc:374] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.129779 28656 raft_consensus.cc:729] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.129794 28655 raft_consensus.cc:729] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.129818 28656 consensus_queue.cc:260] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.129822 28655 consensus_queue.cc:260] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.129894 28656 ts_tablet_manager.cc:1398] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.129920 28655 ts_tablet_manager.cc:1398] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.129959 28656 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.129972 28655 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.130307 28656 tablet_bootstrap.cc:654] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.130306 28655 tablet_bootstrap.cc:654] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.130970 28661 ts_tablet_manager.cc:1888] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.131021 28664 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.131070 28664 ts_tablet_manager.cc:1367] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:18.131186 28664 raft_consensus.cc:348] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.131217 28656 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.131242 28664 raft_consensus.cc:374] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.131249 28656 ts_tablet_manager.cc:1367] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.131260 28664 raft_consensus.cc:729] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.131317 28664 consensus_queue.cc:260] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.131356 28656 raft_consensus.cc:348] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.131410 28656 raft_consensus.cc:374] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.131429 28656 raft_consensus.cc:729] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.131460 28656 consensus_queue.cc:260] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.131474 28664 ts_tablet_manager.cc:1398] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.131520 28664 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.131536 28656 ts_tablet_manager.cc:1398] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.132686 28664 tablet_bootstrap.cc:654] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.132896 28655 tablet_bootstrap.cc:492] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.132937 28655 ts_tablet_manager.cc:1367] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:18.132879 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_58753d33f9dd4a27938992d362e69e56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.132992 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58753d33f9dd4a27938992d362e69e56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.133054 28655 raft_consensus.cc:348] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.133102 28655 raft_consensus.cc:374] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.133117 28655 raft_consensus.cc:729] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.133152 28655 consensus_queue.cc:260] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.133221 28655 ts_tablet_manager.cc:1398] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.133272 28655 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.133581 28664 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.133625 28664 ts_tablet_manager.cc:1367] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:18.133648 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.133718 28655 tablet_bootstrap.cc:654] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.133742 28664 raft_consensus.cc:348] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.133793 28664 raft_consensus.cc:374] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.133810 28664 raft_consensus.cc:729] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.133849 28664 consensus_queue.cc:260] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.133942 28664 ts_tablet_manager.cc:1398] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.133991 28664 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.133989 28710 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20240503 02:09:18.134001 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.134141 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f62c59d2e24782ba20fb8d4f8bec03 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.134289 28664 tablet_bootstrap.cc:654] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.134567 28654 tablet_replica.cc:330] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.134620 28654 raft_consensus.cc:2229] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.134656 28654 raft_consensus.cc:2258] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.134712 28653 tablet_replica.cc:330] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.134763 28653 raft_consensus.cc:2229] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.134794 28653 raft_consensus.cc:2258] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.135504 28653 ts_tablet_manager.cc:1875] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.135860 29550 consensus_queue.cc:1035] T e51a7970b0e147358dab5f6814502ef0 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.135879 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3cbd9d3dfc744f60a112172d69ea0cc0 [id=5ff15871699b4869b63a0187224a6cd7] alter complete (version 4)
I20240503 02:09:18.136827 28653 ts_tablet_manager.cc:1888] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.136876 28653 log.cc:1196] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f2f62c59d2e24782ba20fb8d4f8bec03
I20240503 02:09:18.137063 28653 ts_tablet_manager.cc:1909] T f2f62c59d2e24782ba20fb8d4f8bec03 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.137698 28654 ts_tablet_manager.cc:1875] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.137810 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:18.131013 28661 log.cc:1196] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f2f62c59d2e24782ba20fb8d4f8bec03
I20240503 02:09:18.138697 28661 ts_tablet_manager.cc:1909] T f2f62c59d2e24782ba20fb8d4f8bec03 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.138976 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:18.139068 28654 ts_tablet_manager.cc:1888] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.139101 28654 log.cc:1196] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f2f62c59d2e24782ba20fb8d4f8bec03
I20240503 02:09:18.139232 28654 ts_tablet_manager.cc:1909] T f2f62c59d2e24782ba20fb8d4f8bec03 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.139634 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f2f62c59d2e24782ba20fb8d4f8bec03 (table default.Table_3e4e78e5e04446d2b2287cf636ec1e5f [id=5f0720291c48410aa78697c47dd2831b]) successfully deleted
I20240503 02:09:18.140431 29240 raft_consensus.cc:482] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.140480 29240 raft_consensus.cc:504] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.140498 27554 tablet_service.cc:1467] Processing CreateTablet for tablet c6179ea574834eeb94ae97cbc8cc87c5 (DEFAULT_TABLE table=default.Table_58753d33f9dd4a27938992d362e69e56 [id=87843d5bb9144196909fa97252c232e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.140540 27702 tablet_service.cc:1467] Processing CreateTablet for tablet c6179ea574834eeb94ae97cbc8cc87c5 (DEFAULT_TABLE table=default.Table_58753d33f9dd4a27938992d362e69e56 [id=87843d5bb9144196909fa97252c232e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.140602 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6179ea574834eeb94ae97cbc8cc87c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.140622 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6179ea574834eeb94ae97cbc8cc87c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.140589 29240 leader_election.cc:288] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.141388 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24e57083d8f4dd9b450b0c78a611e0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.141463 27451 raft_consensus.cc:2454] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.141548 27524 leader_election.cc:302] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.141609 29240 raft_consensus.cc:2781] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.141630 29240 raft_consensus.cc:482] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.141646 29240 raft_consensus.cc:3037] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.141747 28656 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.142091 28656 tablet_bootstrap.cc:654] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.142217 29240 raft_consensus.cc:504] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.142330 29240 leader_election.cc:288] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.142305 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3acc4505a58544e983dddb84ffdceac1" } new_table_name: "default.Table_0deba2ee19ce42d397b1107f311967b9" modify_external_catalogs: true
I20240503 02:09:18.142372 27426 tablet_service.cc:1467] Processing CreateTablet for tablet c6179ea574834eeb94ae97cbc8cc87c5 (DEFAULT_TABLE table=default.Table_58753d33f9dd4a27938992d362e69e56 [id=87843d5bb9144196909fa97252c232e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.142463 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6179ea574834eeb94ae97cbc8cc87c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.142485 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24e57083d8f4dd9b450b0c78a611e0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.142536 27719 raft_consensus.cc:3037] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.142594 28656 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.142632 28656 ts_tablet_manager.cc:1367] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.142748 28656 raft_consensus.cc:348] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.142803 28656 raft_consensus.cc:374] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.142820 28656 raft_consensus.cc:729] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.142864 28656 consensus_queue.cc:260] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.143059 28656 ts_tablet_manager.cc:1398] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.143108 28656 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.143105 27719 raft_consensus.cc:2454] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.143455 28656 tablet_bootstrap.cc:654] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.143498 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24e57083d8f4dd9b450b0c78a611e0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.143541 27523 leader_election.cc:302] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.143554 27720 raft_consensus.cc:2362] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:18.143618 27848 raft_consensus.cc:2781] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.143620 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_24404d0b2efa43feb938c7ffd004e375" } new_table_name: "default.Table_228883249382448caaafb6ceae36a539" modify_external_catalogs: true
I20240503 02:09:18.143651 27848 raft_consensus.cc:686] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.143687 27848 consensus_queue.cc:237] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.143769 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24e57083d8f4dd9b450b0c78a611e0f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.143818 27447 raft_consensus.cc:3037] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.144394 27447 raft_consensus.cc:2454] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.144611 28664 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.144662 28664 ts_tablet_manager.cc:1367] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240503 02:09:18.144778 28664 raft_consensus.cc:348] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.144848 28664 raft_consensus.cc:374] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.144867 28664 raft_consensus.cc:729] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.144898 28655 tablet_bootstrap.cc:492] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.144932 28655 ts_tablet_manager.cc:1367] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:18.144917 28664 consensus_queue.cc:260] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.145028 28664 ts_tablet_manager.cc:1398] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.145066 28655 raft_consensus.cc:348] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.145114 28664 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.145119 28655 raft_consensus.cc:374] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.145138 28655 raft_consensus.cc:729] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.145182 28655 consensus_queue.cc:260] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.145280 28655 ts_tablet_manager.cc:1398] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.145370 28655 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.145730 28655 tablet_bootstrap.cc:654] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.146193 28655 tablet_bootstrap.cc:492] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.146242 28655 ts_tablet_manager.cc:1367] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.146363 28655 raft_consensus.cc:348] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.146430 28655 raft_consensus.cc:374] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.146447 28655 raft_consensus.cc:729] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.146484 28655 consensus_queue.cc:260] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.146572 28655 ts_tablet_manager.cc:1398] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.146623 28655 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.146942 28655 tablet_bootstrap.cc:654] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.147439 28655 tablet_bootstrap.cc:492] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.147480 28655 ts_tablet_manager.cc:1367] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.147605 28655 raft_consensus.cc:348] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.147655 28655 raft_consensus.cc:374] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.147671 28655 raft_consensus.cc:729] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.147706 28655 consensus_queue.cc:260] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.147776 28655 ts_tablet_manager.cc:1398] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.147817 28655 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.147936 28664 tablet_bootstrap.cc:654] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.148144 28655 tablet_bootstrap.cc:654] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.148838 28656 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.148880 28656 ts_tablet_manager.cc:1367] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.149003 28656 raft_consensus.cc:348] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.149055 28656 raft_consensus.cc:374] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.149070 28656 raft_consensus.cc:729] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.149111 28656 consensus_queue.cc:260] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.149199 28656 ts_tablet_manager.cc:1398] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.149158 27195 catalog_manager.cc:5484] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.149241 28655 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.149253 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.149272 28655 ts_tablet_manager.cc:1367] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.149435 28655 raft_consensus.cc:348] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.149487 28655 raft_consensus.cc:374] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.149502 28655 raft_consensus.cc:729] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.149492 27582 raft_consensus.cc:1261] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.149536 28655 consensus_queue.cc:260] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.149626 28655 ts_tablet_manager.cc:1398] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.149664 28664 tablet_bootstrap.cc:492] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.149670 28655 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.149698 28664 ts_tablet_manager.cc:1367] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.149816 28664 raft_consensus.cc:348] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.149863 28664 raft_consensus.cc:374] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.149880 28664 raft_consensus.cc:729] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.149927 28664 consensus_queue.cc:260] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.150002 28655 tablet_bootstrap.cc:654] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.150031 28664 ts_tablet_manager.cc:1398] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.150058 27447 raft_consensus.cc:1261] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.150091 28664 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.150514 28664 tablet_bootstrap.cc:654] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.150694 28774 consensus_queue.cc:1035] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.151074 28774 consensus_queue.cc:1035] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.151621 28655 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.151661 28655 ts_tablet_manager.cc:1367] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.151772 28655 raft_consensus.cc:348] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.151822 28655 raft_consensus.cc:374] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.151839 28655 raft_consensus.cc:729] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.151875 28655 consensus_queue.cc:260] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.151959 28655 ts_tablet_manager.cc:1398] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.152014 28655 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.152303 28655 tablet_bootstrap.cc:654] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.152909 30190 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.153157 28710 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.153184 30097 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.153257 28507 consensus_queue.cc:1035] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.153545 30170 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.155871 28410 consensus_queue.cc:1035] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.155917 28664 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.155957 28664 ts_tablet_manager.cc:1367] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.156085 28664 raft_consensus.cc:348] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.156159 28664 raft_consensus.cc:374] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.156178 28664 raft_consensus.cc:729] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.156216 28664 consensus_queue.cc:260] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.156302 28664 ts_tablet_manager.cc:1398] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.156353 28664 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.156692 28664 tablet_bootstrap.cc:654] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.157588 28664 tablet_bootstrap.cc:492] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.157604 28655 tablet_bootstrap.cc:492] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.157634 28664 ts_tablet_manager.cc:1367] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.157639 28655 ts_tablet_manager.cc:1367] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:18.157701 27848 raft_consensus.cc:482] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.157748 27848 raft_consensus.cc:504] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.157756 28664 raft_consensus.cc:348] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.157806 28664 raft_consensus.cc:374] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.157780 28655 raft_consensus.cc:348] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.157824 28664 raft_consensus.cc:729] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.157830 28655 raft_consensus.cc:374] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.157848 28655 raft_consensus.cc:729] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.157845 27848 leader_election.cc:288] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.157866 28664 consensus_queue.cc:260] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.157883 28655 consensus_queue.cc:260] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.157976 28655 ts_tablet_manager.cc:1398] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.157980 28664 ts_tablet_manager.cc:1398] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.157995 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.158031 28664 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.158030 28655 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
W20240503 02:09:18.158181 27524 leader_election.cc:341] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.158330 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet f4b168c17ac44030a73596e9d79123a7 (table default.Table_0deba2ee19ce42d397b1107f311967b9 [id=40a840fc5b274a579372794fd3a3a60d]). Expected version 1 got 0
I20240503 02:09:18.158358 28664 tablet_bootstrap.cc:654] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.158458 28655 tablet_bootstrap.cc:654] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.158944 30097 tablet.cc:1722] T 19e5d244a9364cd595a1089c421d9a06 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.158962 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
W20240503 02:09:18.159199 27523 leader_election.cc:341] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.159245 27523 leader_election.cc:302] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.159405 27853 consensus_queue.cc:1035] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.159665 28410 raft_consensus.cc:2726] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.160246 27853 raft_consensus.cc:482] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.160295 27853 raft_consensus.cc:504] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.160387 27853 leader_election.cc:288] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.160669 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_0deba2ee19ce42d397b1107f311967b9 [id=40a840fc5b274a579372794fd3a3a60d] alter complete (version 1)
I20240503 02:09:18.160655 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633faa7ea971417dbb6116cb7dcb3ef3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.161108 27447 raft_consensus.cc:2454] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.161257 27659 leader_election.cc:302] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.161370 27447 raft_consensus.cc:3037] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.161429 29643 raft_consensus.cc:2781] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.161463 29643 raft_consensus.cc:482] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.161479 29643 raft_consensus.cc:3037] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.161851 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633faa7ea971417dbb6116cb7dcb3ef3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.161911 27588 raft_consensus.cc:2454] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.162133 29643 raft_consensus.cc:504] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.162214 29643 leader_election.cc:288] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.162356 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633faa7ea971417dbb6116cb7dcb3ef3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.162411 27588 raft_consensus.cc:3037] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.162495 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633faa7ea971417dbb6116cb7dcb3ef3" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.162567 27451 raft_consensus.cc:3037] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.163017 28664 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.163055 28664 ts_tablet_manager.cc:1367] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.163072 27588 raft_consensus.cc:2454] T 633faa7ea971417dbb6116cb7dcb3ef3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.163172 30190 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.163205 28664 raft_consensus.cc:348] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.163256 28664 raft_consensus.cc:374] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.163269 27656 leader_election.cc:302] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.163367 28740 raft_consensus.cc:2781] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.163401 28740 raft_consensus.cc:686] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.163437 28740 consensus_queue.cc:237] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.163282 28664 raft_consensus.cc:729] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.163715 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_228883249382448caaafb6ceae36a539 [id=80cf6485f989436d8fab264d6a5743b4] alter complete (version 3)
I20240503 02:09:18.163758 28664 consensus_queue.cc:260] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.163866 28664 ts_tablet_manager.cc:1398] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.164256 29550 raft_consensus.cc:482] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.164305 29550 raft_consensus.cc:504] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.164417 29550 leader_election.cc:288] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.164827 29550 raft_consensus.cc:482] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.164887 29550 raft_consensus.cc:504] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.164965 29550 leader_election.cc:288] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.165088 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.165426 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c934fb82221c45bab114ff8ac97da2bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.165491 27719 raft_consensus.cc:2454] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.165532 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d5f5d7b273241e097bb196744e6f566" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.165582 27582 raft_consensus.cc:2379] T 0d5f5d7b273241e097bb196744e6f566 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:18.165688 27386 leader_election.cc:302] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.165686 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c934fb82221c45bab114ff8ac97da2bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.165743 27586 raft_consensus.cc:2454] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.165771 29550 raft_consensus.cc:3037] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.165911 27386 leader_election.cc:302] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.166013 28766 raft_consensus.cc:2781] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.166038 28766 raft_consensus.cc:482] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.166054 28766 raft_consensus.cc:3037] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.166008 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0deba2ee19ce42d397b1107f311967b9" } new_table_name: "default.Table_7fd195b20d094216ac02c511dba6a326" modify_external_catalogs: true
I20240503 02:09:18.168494 27195 catalog_manager.cc:5484] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.170717 28740 consensus_queue.cc:1035] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.170907 30097 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.172331 30190 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.173592 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0a20b196ff32424d97adbd7278fb3760" } modify_external_catalogs: true
I20240503 02:09:18.173660 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0a20b196ff32424d97adbd7278fb3760" } modify_external_catalogs: true
I20240503 02:09:18.174134 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_7fd195b20d094216ac02c511dba6a326 [id=40a840fc5b274a579372794fd3a3a60d] alter complete (version 2)
I20240503 02:09:18.175020 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b832fd89187a4f9c8e30d646c0541986
I20240503 02:09:18.175282 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet b832fd89187a4f9c8e30d646c0541986 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.175298 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet b832fd89187a4f9c8e30d646c0541986 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.175313 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet b832fd89187a4f9c8e30d646c0541986 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.175366 28661 tablet_replica.cc:330] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.175382 28653 tablet_replica.cc:330] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.175423 28661 raft_consensus.cc:2229] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.175434 28653 raft_consensus.cc:2229] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.175465 28661 raft_consensus.cc:2258] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.175495 28653 raft_consensus.cc:2258] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.175604 28653 ts_tablet_manager.cc:1875] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.175606 28661 ts_tablet_manager.cc:1875] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.177037 28654 tablet_replica.cc:330] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.177049 28693 raft_consensus.cc:482] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.177109 28654 raft_consensus.cc:2229] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.177098 28661 ts_tablet_manager.cc:1888] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.177141 28654 raft_consensus.cc:2258] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.177157 28661 log.cc:1196] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b832fd89187a4f9c8e30d646c0541986
I20240503 02:09:18.177098 28693 raft_consensus.cc:504] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.177243 28654 ts_tablet_manager.cc:1875] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.177248 28693 leader_election.cc:288] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.177387 28661 ts_tablet_manager.cc:1909] T b832fd89187a4f9c8e30d646c0541986 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.177451 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7ccfcbdc854a09b17f6b03897ec63f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.177507 27453 raft_consensus.cc:2454] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.177623 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7ccfcbdc854a09b17f6b03897ec63f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.177676 27586 raft_consensus.cc:2454] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.177752 27659 leader_election.cc:302] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.177839 28693 raft_consensus.cc:2781] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.177881 28693 raft_consensus.cc:482] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.177899 28693 raft_consensus.cc:3037] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.178102 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b832fd89187a4f9c8e30d646c0541986 (table default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]) successfully deleted
I20240503 02:09:18.178419 28693 raft_consensus.cc:504] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.178503 28654 ts_tablet_manager.cc:1888] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.178527 28693 leader_election.cc:288] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.178550 28654 log.cc:1196] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b832fd89187a4f9c8e30d646c0541986
I20240503 02:09:18.178694 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7ccfcbdc854a09b17f6b03897ec63f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.178751 27453 raft_consensus.cc:3037] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.178750 28654 ts_tablet_manager.cc:1909] T b832fd89187a4f9c8e30d646c0541986 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.179332 27453 raft_consensus.cc:2454] T 6f7ccfcbdc854a09b17f6b03897ec63f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.179738 28653 ts_tablet_manager.cc:1888] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.179726 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f7ccfcbdc854a09b17f6b03897ec63f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.179762 27659 leader_election.cc:302] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.179785 27586 raft_consensus.cc:3037] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.179790 28653 log.cc:1196] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b832fd89187a4f9c8e30d646c0541986
I20240503 02:09:18.179975 28653 ts_tablet_manager.cc:1909] T b832fd89187a4f9c8e30d646c0541986 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.180229 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b832fd89187a4f9c8e30d646c0541986 (table default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]) successfully deleted
I20240503 02:09:18.180343 27586 raft_consensus.cc:2454] T 6f7ccfcbdc854a09b17f6b03897ec63f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.180406 27848 raft_consensus.cc:482] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.180420 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b832fd89187a4f9c8e30d646c0541986 (table default.Table_0a20b196ff32424d97adbd7278fb3760 [id=8205a8b404794e4ba2d16059063466a1]) successfully deleted
I20240503 02:09:18.180447 27848 raft_consensus.cc:504] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.180533 27848 leader_election.cc:288] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.180581 29643 raft_consensus.cc:2781] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.180625 29643 raft_consensus.cc:686] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.180661 29643 consensus_queue.cc:237] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.181118 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc8f22f822e4b3e829bba41bd702f0c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.181197 27719 raft_consensus.cc:2454] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.181316 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc8f22f822e4b3e829bba41bd702f0c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.181393 27453 raft_consensus.cc:2454] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.181442 27523 leader_election.cc:302] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.181511 27848 raft_consensus.cc:2781] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.181542 27848 raft_consensus.cc:482] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.181560 27848 raft_consensus.cc:3037] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.181730 27200 catalog_manager.cc:5484] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.182129 27848 raft_consensus.cc:504] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.182227 27848 leader_election.cc:288] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.182500 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc8f22f822e4b3e829bba41bd702f0c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.182518 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc8f22f822e4b3e829bba41bd702f0c" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.182554 27453 raft_consensus.cc:3037] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.182574 27719 raft_consensus.cc:3037] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.182739 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9957972e38de45fb97dc8e24a14d8314" } modify_external_catalogs: true
I20240503 02:09:18.182791 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9957972e38de45fb97dc8e24a14d8314" } modify_external_catalogs: true
I20240503 02:09:18.183153 27453 raft_consensus.cc:2454] T dcc8f22f822e4b3e829bba41bd702f0c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.183164 27719 raft_consensus.cc:2454] T dcc8f22f822e4b3e829bba41bd702f0c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.183276 27524 leader_election.cc:302] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.183322 29643 consensus_queue.cc:1035] T 389839158b314fd7a34852af7b4afadc P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.183406 27848 raft_consensus.cc:2781] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.183457 27848 raft_consensus.cc:686] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.183493 27848 consensus_queue.cc:237] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.184159 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 7a23e797aee94bcb90cbb7545147a893
I20240503 02:09:18.184386 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a23e797aee94bcb90cbb7545147a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.184417 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a23e797aee94bcb90cbb7545147a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.184646 28653 tablet_replica.cc:330] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.184713 28653 raft_consensus.cc:2229] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.184742 28653 raft_consensus.cc:2258] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.184804 28654 tablet_replica.cc:330] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.184856 28654 raft_consensus.cc:2229] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.184903 28654 pending_rounds.cc:62] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:18.184924 28654 pending_rounds.cc:68] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420039439740928 op_type: NO_OP noop_request { }
I20240503 02:09:18.184931 28653 ts_tablet_manager.cc:1875] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.184955 28654 raft_consensus.cc:2866] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.184978 28654 raft_consensus.cc:2258] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.186281 28653 ts_tablet_manager.cc:1888] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.186326 28653 log.cc:1196] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/7a23e797aee94bcb90cbb7545147a893
I20240503 02:09:18.186470 28653 ts_tablet_manager.cc:1909] T 7a23e797aee94bcb90cbb7545147a893 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.186923 28693 consensus_queue.cc:1035] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.186987 29550 raft_consensus.cc:2726] T 0d5f5d7b273241e097bb196744e6f566 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.187738 27451 raft_consensus.cc:2454] T 633faa7ea971417dbb6116cb7dcb3ef3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.187808 28655 tablet_bootstrap.cc:492] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.187855 28655 ts_tablet_manager.cc:1367] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.001s
I20240503 02:09:18.188004 28655 raft_consensus.cc:348] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.188053 28655 raft_consensus.cc:374] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.188064 28655 raft_consensus.cc:729] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.188102 28655 consensus_queue.cc:260] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.188189 28655 ts_tablet_manager.cc:1398] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.188285 29550 raft_consensus.cc:482] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.188336 29550 raft_consensus.cc:504] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.188412 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet b0d7641f83dc4d029ecd170a2d14888b from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.188438 29550 leader_election.cc:288] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.188568 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b24a6595e7ea4b13a2cb7e5894bdc5d3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.188622 27719 raft_consensus.cc:2454] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.188724 27388 leader_election.cc:302] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.188870 29550 raft_consensus.cc:2781] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.188905 29550 raft_consensus.cc:482] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.188922 29550 raft_consensus.cc:3037] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.189530 29550 raft_consensus.cc:504] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.189651 29550 leader_election.cc:288] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.190013 28693 raft_consensus.cc:482] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.190060 28693 raft_consensus.cc:504] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.190127 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b24a6595e7ea4b13a2cb7e5894bdc5d3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.190192 27719 raft_consensus.cc:3037] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.190609 30170 tablet.cc:1722] T f4b168c17ac44030a73596e9d79123a7 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.190683 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15819b1d3eec4d6fb3d1ef884508370a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.190728 27453 raft_consensus.cc:2454] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.190699 27197 catalog_manager.cc:5484] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.190783 27719 raft_consensus.cc:2454] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.190889 27659 leader_election.cc:302] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.190913 27388 leader_election.cc:302] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.190977 28774 raft_consensus.cc:2781] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.190904 27853 consensus_queue.cc:1035] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.191004 28774 raft_consensus.cc:482] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.191123 28774 raft_consensus.cc:3037] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.190143 28693 leader_election.cc:288] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.191288 29550 raft_consensus.cc:2781] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.191303 28654 ts_tablet_manager.cc:1875] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.191326 29550 raft_consensus.cc:686] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.191361 29550 consensus_queue.cc:237] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.191570 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b0d7641f83dc4d029ecd170a2d14888b
I20240503 02:09:18.191641 27200 catalog_manager.cc:6554] ReplaceTablet: tablet b0d7641f83dc4d029ecd170a2d14888b deleted and replaced by tablet 97721d6cf071459baaa04b58a36c7013
I20240503 02:09:18.191699 28774 raft_consensus.cc:504] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.191797 28774 leader_election.cc:288] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.191855 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d7641f83dc4d029ecd170a2d14888b with delete_type TABLET_DATA_DELETED (replaced by tablet 97721d6cf071459baaa04b58a36c7013) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.192153 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d7641f83dc4d029ecd170a2d14888b with delete_type TABLET_DATA_DELETED (replaced by tablet 97721d6cf071459baaa04b58a36c7013) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.192543 28654 ts_tablet_manager.cc:1888] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.192607 28654 log.cc:1196] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/7a23e797aee94bcb90cbb7545147a893
I20240503 02:09:18.192786 28654 ts_tablet_manager.cc:1909] T 7a23e797aee94bcb90cbb7545147a893 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.192919 28766 raft_consensus.cc:504] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.193040 28766 leader_election.cc:288] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.193390 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15819b1d3eec4d6fb3d1ef884508370a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.193437 27584 raft_consensus.cc:3037] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.193491 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c934fb82221c45bab114ff8ac97da2bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.193536 27719 raft_consensus.cc:3037] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.194059 27584 raft_consensus.cc:2454] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.194096 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b24a6595e7ea4b13a2cb7e5894bdc5d3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.194121 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b24a6595e7ea4b13a2cb7e5894bdc5d3" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.194155 27586 raft_consensus.cc:2454] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.193281 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15819b1d3eec4d6fb3d1ef884508370a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.194295 27585 raft_consensus.cc:2362] T 15819b1d3eec4d6fb3d1ef884508370a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:18.194485 27719 raft_consensus.cc:2454] T c934fb82221c45bab114ff8ac97da2bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.194644 27388 leader_election.cc:302] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.194715 29551 raft_consensus.cc:2781] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.194754 29551 raft_consensus.cc:686] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.194797 29551 consensus_queue.cc:237] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.195225 28653 tablet_replica.cc:330] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.195289 28653 raft_consensus.cc:2229] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.195338 28653 raft_consensus.cc:2258] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.195425 28653 ts_tablet_manager.cc:1875] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.196305 28654 tablet_replica.cc:330] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.196388 28654 raft_consensus.cc:2229] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.196401 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 7a23e797aee94bcb90cbb7545147a893 (table default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]) successfully deleted
I20240503 02:09:18.196424 28654 raft_consensus.cc:2258] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.196509 28654 ts_tablet_manager.cc:1875] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.197467 28653 ts_tablet_manager.cc:1888] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.197557 27656 leader_election.cc:302] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.197675 28740 raft_consensus.cc:2781] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.197737 28740 raft_consensus.cc:686] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.197769 28740 consensus_queue.cc:237] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.197971 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a23e797aee94bcb90cbb7545147a893 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.197985 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 7a23e797aee94bcb90cbb7545147a893 (table default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]) successfully deleted
I20240503 02:09:18.198029 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d7641f83dc4d029ecd170a2d14888b with delete_type TABLET_DATA_DELETED (replaced by tablet 97721d6cf071459baaa04b58a36c7013) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.198369 28661 tablet_replica.cc:330] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.198438 28661 raft_consensus.cc:2229] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.198468 28661 raft_consensus.cc:2258] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.199409 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet f42c6ab1499d4b058241376b6f211bde from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.199584 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c934fb82221c45bab114ff8ac97da2bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.199637 27585 raft_consensus.cc:3037] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.199733 28661 ts_tablet_manager.cc:1875] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.199787 28774 consensus_queue.cc:1035] T c5707b69d91141ddbc3e9e8d8c3af477 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.200224 27585 raft_consensus.cc:2454] T c934fb82221c45bab114ff8ac97da2bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.201061 28661 ts_tablet_manager.cc:1888] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.201109 28661 log.cc:1196] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/7a23e797aee94bcb90cbb7545147a893
I20240503 02:09:18.201257 28661 ts_tablet_manager.cc:1909] T 7a23e797aee94bcb90cbb7545147a893 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.201313 28654 ts_tablet_manager.cc:1888] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.201380 28654 log.cc:1196] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b0d7641f83dc4d029ecd170a2d14888b
I20240503 02:09:18.201577 28654 ts_tablet_manager.cc:1909] T b0d7641f83dc4d029ecd170a2d14888b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.198825 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15819b1d3eec4d6fb3d1ef884508370a" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.201885 27453 raft_consensus.cc:3037] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.201910 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b0d7641f83dc4d029ecd170a2d14888b (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:18.199173 28766 consensus_queue.cc:1035] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.202545 27453 raft_consensus.cc:2454] T 15819b1d3eec4d6fb3d1ef884508370a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.202913 27453 raft_consensus.cc:1261] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.203369 28410 consensus_queue.cc:1035] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.203481 28653 log.cc:1196] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b0d7641f83dc4d029ecd170a2d14888b
I20240503 02:09:18.203744 28653 ts_tablet_manager.cc:1909] T b0d7641f83dc4d029ecd170a2d14888b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.204238 27200 catalog_manager.cc:5484] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.204428 27195 catalog_manager.cc:5484] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.204492 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 97721d6cf071459baaa04b58a36c7013 (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.204573 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97721d6cf071459baaa04b58a36c7013. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.204902 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 97721d6cf071459baaa04b58a36c7013 (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.204913 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f42c6ab1499d4b058241376b6f211bde
I20240503 02:09:18.204984 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97721d6cf071459baaa04b58a36c7013. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.204985 27197 catalog_manager.cc:6554] ReplaceTablet: tablet f42c6ab1499d4b058241376b6f211bde deleted and replaced by tablet 1a6aabb9dc28424088f42567396efdd7
I20240503 02:09:18.205040 27720 raft_consensus.cc:1261] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.205467 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet f42c6ab1499d4b058241376b6f211bde with delete_type TABLET_DATA_DELETED (replaced by tablet 1a6aabb9dc28424088f42567396efdd7) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.205603 28410 consensus_queue.cc:1035] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.205925 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f42c6ab1499d4b058241376b6f211bde with delete_type TABLET_DATA_DELETED (replaced by tablet 1a6aabb9dc28424088f42567396efdd7) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.206022 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b0d7641f83dc4d029ecd170a2d14888b (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:18.206048 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet f42c6ab1499d4b058241376b6f211bde with delete_type TABLET_DATA_DELETED (replaced by tablet 1a6aabb9dc28424088f42567396efdd7) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.206108 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 97721d6cf071459baaa04b58a36c7013 (DEFAULT_TABLE table=default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.206189 28653 tablet_replica.cc:330] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.206259 28654 tablet_replica.cc:330] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.206346 28653 raft_consensus.cc:2229] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.206445 28656 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.206491 28664 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.206817 28656 tablet_bootstrap.cc:654] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.206898 28664 tablet_bootstrap.cc:654] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.207967 28654 raft_consensus.cc:2229] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.206202 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97721d6cf071459baaa04b58a36c7013. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.208089 28653 raft_consensus.cc:2258] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.208133 28740 raft_consensus.cc:482] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.208186 28740 raft_consensus.cc:504] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.208289 28740 leader_election.cc:288] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.208518 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a551225a4c8f4e4f84410bcb6fa38634" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.208539 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a551225a4c8f4e4f84410bcb6fa38634" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.208577 27447 raft_consensus.cc:2454] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.208596 27586 raft_consensus.cc:2454] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.208701 27659 leader_election.cc:302] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.208716 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6aabb9dc28424088f42567396efdd7 (DEFAULT_TABLE table=default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.208801 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6aabb9dc28424088f42567396efdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.208878 28740 raft_consensus.cc:2781] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.208910 28740 raft_consensus.cc:482] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.208925 28740 raft_consensus.cc:3037] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.209030 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.209084 27197 catalog_manager.cc:5484] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.207458 28766 raft_consensus.cc:482] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.209233 28655 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.209265 28766 raft_consensus.cc:504] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.209398 28766 leader_election.cc:288] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.209614 28655 tablet_bootstrap.cc:654] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.209595 28740 raft_consensus.cc:504] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.209698 28740 leader_election.cc:288] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.209906 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6aabb9dc28424088f42567396efdd7 (DEFAULT_TABLE table=default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.210011 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6aabb9dc28424088f42567396efdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.210052 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e7025bcc084d6d9a6d34e61b75d9c7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.210068 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 1a6aabb9dc28424088f42567396efdd7 (DEFAULT_TABLE table=default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.210099 27720 raft_consensus.cc:2454] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.210167 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a6aabb9dc28424088f42567396efdd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.211325 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a551225a4c8f4e4f84410bcb6fa38634" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.211383 27447 raft_consensus.cc:3037] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.211509 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet a24e57083d8f4dd9b450b0c78a611e0f (table default.Table_405719b600a24ad69a35850163c91c9c [id=09f75c45684343a9bae908f60652e20e]). Expected version 2 got 0
I20240503 02:09:18.211932 28661 tablet_replica.cc:330] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.211944 27447 raft_consensus.cc:2454] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.211992 28661 raft_consensus.cc:2229] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.212019 28661 raft_consensus.cc:2258] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.212103 28661 ts_tablet_manager.cc:1875] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.212092 27388 leader_election.cc:302] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.212177 28766 raft_consensus.cc:2781] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.212209 28766 raft_consensus.cc:482] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.212226 28766 raft_consensus.cc:3037] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.212852 28766 raft_consensus.cc:504] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.212945 28766 leader_election.cc:288] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.213035 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e7025bcc084d6d9a6d34e61b75d9c7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.213107 27586 raft_consensus.cc:2454] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.213196 27659 leader_election.cc:302] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.213359 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a551225a4c8f4e4f84410bcb6fa38634" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.213413 27585 raft_consensus.cc:3037] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.213387 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e7025bcc084d6d9a6d34e61b75d9c7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.213443 27582 raft_consensus.cc:3037] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.213480 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0e7025bcc084d6d9a6d34e61b75d9c7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.213526 27719 raft_consensus.cc:3037] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.213982 27585 raft_consensus.cc:2454] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.214042 27582 raft_consensus.cc:2454] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.214042 27719 raft_consensus.cc:2454] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.214341 28774 raft_consensus.cc:2781] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.214363 27388 leader_election.cc:302] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.214390 28774 raft_consensus.cc:686] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.214426 28685 raft_consensus.cc:2781] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.214426 28774 consensus_queue.cc:237] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.214469 28685 raft_consensus.cc:686] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.214500 28685 consensus_queue.cc:237] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.215286 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 7a23e797aee94bcb90cbb7545147a893 (table default.Table_9957972e38de45fb97dc8e24a14d8314 [id=bfe503bccb6b4ad2a61ab3bc37f17d70]) successfully deleted
I20240503 02:09:18.215394 28654 raft_consensus.cc:2258] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:18.215409 27195 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:18.215652 28654 ts_tablet_manager.cc:1875] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.216202 27197 catalog_manager.cc:5484] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
W20240503 02:09:18.216552 27523 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Couldn't send request to peer c2be543b8aad4e80a58d6b4ab669faae. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 11: this message will repeat every 5th retry.
I20240503 02:09:18.216761 28766 consensus_queue.cc:1035] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.217258 27195 catalog_manager.cc:5484] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.217666 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 4c14768b631a423084b6f43203a8832e from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.218196 28654 ts_tablet_manager.cc:1888] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.218238 28654 log.cc:1196] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f42c6ab1499d4b058241376b6f211bde
I20240503 02:09:18.218497 28654 ts_tablet_manager.cc:1909] T f42c6ab1499d4b058241376b6f211bde P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.218690 28653 ts_tablet_manager.cc:1875] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.219101 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4c14768b631a423084b6f43203a8832e
I20240503 02:09:18.219168 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 4c14768b631a423084b6f43203a8832e deleted and replaced by tablet 6540e09a284246ee92301479690ed27c
I20240503 02:09:18.219197 28661 ts_tablet_manager.cc:1888] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.219239 28661 log.cc:1196] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b0d7641f83dc4d029ecd170a2d14888b
I20240503 02:09:18.219326 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c14768b631a423084b6f43203a8832e with delete_type TABLET_DATA_DELETED (replaced by tablet 6540e09a284246ee92301479690ed27c) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.219467 28661 ts_tablet_manager.cc:1909] T b0d7641f83dc4d029ecd170a2d14888b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.219756 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c14768b631a423084b6f43203a8832e with delete_type TABLET_DATA_DELETED (replaced by tablet 6540e09a284246ee92301479690ed27c) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.219831 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c14768b631a423084b6f43203a8832e with delete_type TABLET_DATA_DELETED (replaced by tablet 6540e09a284246ee92301479690ed27c) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.219956 28740 raft_consensus.cc:482] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.219991 28740 raft_consensus.cc:504] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.220075 28740 leader_election.cc:288] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.220299 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e60cbc3e764f71bcfb0ae5d0558fd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.220374 27447 raft_consensus.cc:2454] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.220368 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e60cbc3e764f71bcfb0ae5d0558fd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.220425 27586 raft_consensus.cc:2454] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.220531 27656 leader_election.cc:302] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.220645 28740 raft_consensus.cc:2781] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.220681 28740 raft_consensus.cc:482] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.220705 28740 raft_consensus.cc:3037] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.220803 28653 ts_tablet_manager.cc:1888] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.220836 28653 log.cc:1196] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f42c6ab1499d4b058241376b6f211bde
I20240503 02:09:18.220999 28653 ts_tablet_manager.cc:1909] T f42c6ab1499d4b058241376b6f211bde P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.221218 28661 tablet_replica.cc:330] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.221241 28740 raft_consensus.cc:504] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.221271 28661 raft_consensus.cc:2229] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.221299 28661 raft_consensus.cc:2258] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.221562 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b0d7641f83dc4d029ecd170a2d14888b (table default.Table_e1e149474317465db7cfab2add6adb8b [id=291ce53479964cc19502a7135d99e2f8]) successfully deleted
I20240503 02:09:18.221642 28653 tablet_replica.cc:330] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.221639 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f42c6ab1499d4b058241376b6f211bde (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:18.221700 28653 raft_consensus.cc:2229] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.221732 28653 raft_consensus.cc:2258] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.221753 28740 leader_election.cc:288] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.221956 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e60cbc3e764f71bcfb0ae5d0558fd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.222005 27447 raft_consensus.cc:3037] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.222005 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f42c6ab1499d4b058241376b6f211bde (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:18.222520 27447 raft_consensus.cc:2454] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.222707 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 6540e09a284246ee92301479690ed27c (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.222673 27659 leader_election.cc:302] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.222762 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6540e09a284246ee92301479690ed27c (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.222805 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540e09a284246ee92301479690ed27c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.222811 29643 raft_consensus.cc:2781] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.222842 29643 raft_consensus.cc:686] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.222856 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540e09a284246ee92301479690ed27c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.222874 29643 consensus_queue.cc:237] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.222568 28661 ts_tablet_manager.cc:1875] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.224521 27200 catalog_manager.cc:5484] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.222146 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e60cbc3e764f71bcfb0ae5d0558fd8" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.224655 27586 raft_consensus.cc:3037] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.225265 27586 raft_consensus.cc:2454] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.222339 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 6540e09a284246ee92301479690ed27c (DEFAULT_TABLE table=default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.225464 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540e09a284246ee92301479690ed27c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.225339 28661 ts_tablet_manager.cc:1888] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.225546 28661 log.cc:1196] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f42c6ab1499d4b058241376b6f211bde
I20240503 02:09:18.225714 28661 ts_tablet_manager.cc:1909] T f42c6ab1499d4b058241376b6f211bde P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.225950 28661 tablet_replica.cc:330] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.225962 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f42c6ab1499d4b058241376b6f211bde (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]) successfully deleted
I20240503 02:09:18.226011 28661 raft_consensus.cc:2229] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.226044 28661 raft_consensus.cc:2258] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.226238 28661 ts_tablet_manager.cc:1875] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.221930 28654 tablet_replica.cc:330] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.226855 28654 raft_consensus.cc:2229] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.226928 28654 raft_consensus.cc:2258] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.227067 28653 ts_tablet_manager.cc:1875] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.227427 28661 ts_tablet_manager.cc:1888] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.227563 28661 log.cc:1196] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4c14768b631a423084b6f43203a8832e
I20240503 02:09:18.227748 28661 ts_tablet_manager.cc:1909] T 4c14768b631a423084b6f43203a8832e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.228534 28740 raft_consensus.cc:482] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.228580 28740 raft_consensus.cc:504] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.228641 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 4ca02d87dfd549e9bf01c7f3d38928e3 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.228663 28653 ts_tablet_manager.cc:1888] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.228684 28740 leader_election.cc:288] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.228703 28653 log.cc:1196] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4c14768b631a423084b6f43203a8832e
I20240503 02:09:18.228802 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet acb56205ff0d4a3ba73d21d9e821a43a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.228801 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.228848 27586 raft_consensus.cc:2454] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.228936 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4c14768b631a423084b6f43203a8832e (table default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]) successfully deleted
I20240503 02:09:18.229020 28653 ts_tablet_manager.cc:1909] T 4c14768b631a423084b6f43203a8832e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.229290 28654 ts_tablet_manager.cc:1875] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.229384 27656 leader_election.cc:302] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.229493 27853 raft_consensus.cc:2781] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.229526 27853 raft_consensus.cc:482] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.229513 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4c14768b631a423084b6f43203a8832e (table default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]) successfully deleted
I20240503 02:09:18.229543 27853 raft_consensus.cc:3037] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.229970 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet acb56205ff0d4a3ba73d21d9e821a43a
I20240503 02:09:18.230051 27197 catalog_manager.cc:6554] ReplaceTablet: tablet acb56205ff0d4a3ba73d21d9e821a43a deleted and replaced by tablet 6062b6fee7974681b5fddbea0eb1ecfc
I20240503 02:09:18.230144 27853 raft_consensus.cc:504] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.230178 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a with delete_type TABLET_DATA_DELETED (replaced by tablet 6062b6fee7974681b5fddbea0eb1ecfc) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.230238 27853 leader_election.cc:288] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.230271 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a with delete_type TABLET_DATA_DELETED (replaced by tablet 6062b6fee7974681b5fddbea0eb1ecfc) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.230734 28654 ts_tablet_manager.cc:1888] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.230760 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 4ca02d87dfd549e9bf01c7f3d38928e3
I20240503 02:09:18.230837 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 4ca02d87dfd549e9bf01c7f3d38928e3 deleted and replaced by tablet 42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.230921 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.230986 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.231021 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6179ea574834eeb94ae97cbc8cc87c5" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.231066 27586 raft_consensus.cc:3037] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.230989 27447 raft_consensus.cc:3037] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.231652 27586 raft_consensus.cc:2454] T c6179ea574834eeb94ae97cbc8cc87c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.231770 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 42e6c708a5fd43c09fb1231ec0a1d17e) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.231777 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet acb56205ff0d4a3ba73d21d9e821a43a with delete_type TABLET_DATA_DELETED (replaced by tablet 6062b6fee7974681b5fddbea0eb1ecfc) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.231858 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 42e6c708a5fd43c09fb1231ec0a1d17e) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.232023 27447 raft_consensus.cc:2454] T c6179ea574834eeb94ae97cbc8cc87c5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.232470 27659 leader_election.cc:302] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.232514 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca02d87dfd549e9bf01c7f3d38928e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 42e6c708a5fd43c09fb1231ec0a1d17e) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.232563 29643 raft_consensus.cc:2781] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.232599 29643 raft_consensus.cc:686] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.230777 28654 log.cc:1196] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4c14768b631a423084b6f43203a8832e
I20240503 02:09:18.232646 29643 consensus_queue.cc:237] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.232884 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 6062b6fee7974681b5fddbea0eb1ecfc (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.232976 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6062b6fee7974681b5fddbea0eb1ecfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.234658 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.234743 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e6c708a5fd43c09fb1231ec0a1d17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.234980 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.235003 27200 catalog_manager.cc:5484] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.235067 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e6c708a5fd43c09fb1231ec0a1d17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.235260 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 6062b6fee7974681b5fddbea0eb1ecfc (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.235347 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6062b6fee7974681b5fddbea0eb1ecfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.232722 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 6062b6fee7974681b5fddbea0eb1ecfc (DEFAULT_TABLE table=default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.234009 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e (DEFAULT_TABLE table=default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.235858 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6062b6fee7974681b5fddbea0eb1ecfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.235944 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e6c708a5fd43c09fb1231ec0a1d17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.236413 29551 raft_consensus.cc:482] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.236454 29551 raft_consensus.cc:504] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.236546 29551 leader_election.cc:288] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.236936 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.237125 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed2da494a334cbab2f24ca578adc494" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.237188 27719 raft_consensus.cc:2379] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:18.237293 30190 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.237350 27388 leader_election.cc:302] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.237394 28661 tablet_replica.cc:330] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.237429 29551 raft_consensus.cc:3037] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.237452 28661 raft_consensus.cc:2229] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.237481 28661 raft_consensus.cc:2258] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.237638 28661 ts_tablet_manager.cc:1875] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.237944 28653 tablet_replica.cc:330] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.238003 28653 raft_consensus.cc:2229] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.238034 28653 raft_consensus.cc:2258] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.238051 29551 raft_consensus.cc:2726] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.238241 30097 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.238282 29240 raft_consensus.cc:482] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.238327 29240 raft_consensus.cc:504] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.238413 29240 leader_election.cc:288] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.239079 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b363bfa5d6f04d658cec669b62811c4f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.239109 28661 ts_tablet_manager.cc:1888] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.239136 27719 raft_consensus.cc:2454] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.239153 28661 log.cc:1196] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/4ca02d87dfd549e9bf01c7f3d38928e3
I20240503 02:09:18.239295 28661 ts_tablet_manager.cc:1909] T 4ca02d87dfd549e9bf01c7f3d38928e3 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.239444 28653 ts_tablet_manager.cc:1875] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.239632 28661 tablet_replica.cc:330] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.239737 28661 raft_consensus.cc:2229] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.239771 28661 raft_consensus.cc:2258] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.240021 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.240610 27523 leader_election.cc:302] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.240787 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b363bfa5d6f04d658cec669b62811c4f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.240849 27447 raft_consensus.cc:2454] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.240931 28653 ts_tablet_manager.cc:1888] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.241003 28653 log.cc:1196] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/acb56205ff0d4a3ba73d21d9e821a43a
I20240503 02:09:18.237295 28654 ts_tablet_manager.cc:1909] T 4c14768b631a423084b6f43203a8832e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.241163 28653 ts_tablet_manager.cc:1909] T acb56205ff0d4a3ba73d21d9e821a43a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.241416 28654 tablet_replica.cc:330] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.241447 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4c14768b631a423084b6f43203a8832e (table default.Table_2b3a7371a1ac4f049968c590a81af503 [id=3f8a55dd9ed14a28a8cbf650b3375eda]) successfully deleted
I20240503 02:09:18.241595 28653 tablet_replica.cc:330] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.241640 29240 raft_consensus.cc:2781] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.241693 29240 raft_consensus.cc:482] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.241714 29240 raft_consensus.cc:3037] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.241773 28653 raft_consensus.cc:2229] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.241907 28653 raft_consensus.cc:2258] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.242106 28653 ts_tablet_manager.cc:1875] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.242215 29240 raft_consensus.cc:504] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.242331 29240 leader_election.cc:288] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.242491 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b363bfa5d6f04d658cec669b62811c4f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.242489 27448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b363bfa5d6f04d658cec669b62811c4f" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.242551 27719 raft_consensus.cc:3037] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.242565 27448 raft_consensus.cc:3037] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.242767 29551 raft_consensus.cc:482] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.242805 29551 raft_consensus.cc:504] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.242882 29551 leader_election.cc:288] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.243023 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d79bb60500d483985daa3ab36380409" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.243077 27718 raft_consensus.cc:2454] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.243103 27719 raft_consensus.cc:2454] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.243161 27448 raft_consensus.cc:2454] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.243208 27388 leader_election.cc:302] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.243234 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d79bb60500d483985daa3ab36380409" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.243283 27586 raft_consensus.cc:2454] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.243256 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a604d0eec20e4037ad895d3802e5341f" } modify_external_catalogs: true
I20240503 02:09:18.243304 27523 leader_election.cc:302] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.243315 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a604d0eec20e4037ad895d3802e5341f" } modify_external_catalogs: true
I20240503 02:09:18.243407 27848 raft_consensus.cc:2781] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.243453 27848 raft_consensus.cc:686] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.243494 27848 consensus_queue.cc:237] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.243541 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet acb56205ff0d4a3ba73d21d9e821a43a (table default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]) successfully deleted
I20240503 02:09:18.244284 29551 raft_consensus.cc:2781] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.244333 29551 raft_consensus.cc:482] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.244352 29551 raft_consensus.cc:3037] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.244563 28661 ts_tablet_manager.cc:1875] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.244693 28653 ts_tablet_manager.cc:1888] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.244769 28653 log.cc:1196] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/4ca02d87dfd549e9bf01c7f3d38928e3
I20240503 02:09:18.244942 29551 raft_consensus.cc:504] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.244975 28653 ts_tablet_manager.cc:1909] T 4ca02d87dfd549e9bf01c7f3d38928e3 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.245066 29551 leader_election.cc:288] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.245232 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d79bb60500d483985daa3ab36380409" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.245286 27586 raft_consensus.cc:3037] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.245313 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d79bb60500d483985daa3ab36380409" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.245383 27718 raft_consensus.cc:3037] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.245497 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 2081f9c133424955a57f3b9362c8fbd8
I20240503 02:09:18.245812 27195 catalog_manager.cc:5484] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.245865 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.245894 27586 raft_consensus.cc:2454] T 5d79bb60500d483985daa3ab36380409 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.245908 27718 raft_consensus.cc:2454] T 5d79bb60500d483985daa3ab36380409 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.246024 27388 leader_election.cc:302] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.246109 29551 raft_consensus.cc:2781] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.246155 29551 raft_consensus.cc:686] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.246191 29551 consensus_queue.cc:237] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.246690 28661 ts_tablet_manager.cc:1888] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.246731 28661 log.cc:1196] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/acb56205ff0d4a3ba73d21d9e821a43a
I20240503 02:09:18.246884 28661 ts_tablet_manager.cc:1909] T acb56205ff0d4a3ba73d21d9e821a43a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.247278 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet acb56205ff0d4a3ba73d21d9e821a43a (table default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]) successfully deleted
I20240503 02:09:18.247345 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.247627 28661 tablet_replica.cc:330] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.247725 28661 raft_consensus.cc:2229] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.247813 28661 raft_consensus.cc:2258] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.247992 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 2081f9c133424955a57f3b9362c8fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.248096 28661 ts_tablet_manager.cc:1875] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.248183 28653 tablet_replica.cc:330] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.248246 28653 raft_consensus.cc:2229] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.248284 28653 raft_consensus.cc:2258] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.248265 27195 catalog_manager.cc:5484] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.241628 28654 raft_consensus.cc:2229] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.248795 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_c7ed276358c14b26bfd22e4bf3f709e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.248903 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7ed276358c14b26bfd22e4bf3f709e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.248704 28654 raft_consensus.cc:2258] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.249840 30170 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.250254 28654 ts_tablet_manager.cc:1875] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.250763 28656 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.250811 28656 ts_tablet_manager.cc:1367] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20240503 02:09:18.250949 28656 raft_consensus.cc:348] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.251005 28656 raft_consensus.cc:374] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.251021 28656 raft_consensus.cc:729] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.251065 28656 consensus_queue.cc:260] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.251173 28656 ts_tablet_manager.cc:1398] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.251233 28656 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.251567 28656 tablet_bootstrap.cc:654] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.251709 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.252231 28654 ts_tablet_manager.cc:1888] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.252277 28654 log.cc:1196] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/acb56205ff0d4a3ba73d21d9e821a43a
I20240503 02:09:18.252424 28654 ts_tablet_manager.cc:1909] T acb56205ff0d4a3ba73d21d9e821a43a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.252885 28664 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.252930 28664 ts_tablet_manager.cc:1367] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.000s
I20240503 02:09:18.253057 28664 raft_consensus.cc:348] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.253122 28664 raft_consensus.cc:374] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.253155 28664 raft_consensus.cc:729] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.253191 28664 consensus_queue.cc:260] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.253214 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet acb56205ff0d4a3ba73d21d9e821a43a (table default.Table_eab18f79ec204528b7d1a22f3cbdc354 [id=c76a806872474cbeb79b3219b68a14a1]) successfully deleted
I20240503 02:09:18.253273 28664 ts_tablet_manager.cc:1398] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.253516 28664 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.253861 28664 tablet_bootstrap.cc:654] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.253281 28654 tablet_replica.cc:330] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.254282 28654 raft_consensus.cc:2229] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.254396 28654 pending_rounds.cc:62] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:18.254452 28654 pending_rounds.cc:68] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420038981267456 op_type: NO_OP noop_request { }
I20240503 02:09:18.254529 28654 raft_consensus.cc:2866] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.254595 28654 raft_consensus.cc:2258] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.254985 28653 ts_tablet_manager.cc:1875] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.255470 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd59c2a068642c6af4742c990176172 (DEFAULT_TABLE table=default.Table_c7ed276358c14b26bfd22e4bf3f709e1 [id=146e32948828496a8e05763492a6b34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.255564 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd59c2a068642c6af4742c990176172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.255564 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd59c2a068642c6af4742c990176172 (DEFAULT_TABLE table=default.Table_c7ed276358c14b26bfd22e4bf3f709e1 [id=146e32948828496a8e05763492a6b34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.256305 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd59c2a068642c6af4742c990176172 (DEFAULT_TABLE table=default.Table_c7ed276358c14b26bfd22e4bf3f709e1 [id=146e32948828496a8e05763492a6b34d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.256517 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd59c2a068642c6af4742c990176172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.255456 28654 ts_tablet_manager.cc:1875] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.257928 28653 ts_tablet_manager.cc:1888] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240503 02:09:18.257977 28653 log.cc:1196] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/2081f9c133424955a57f3b9362c8fbd8
I20240503 02:09:18.258123 28653 ts_tablet_manager.cc:1909] T 2081f9c133424955a57f3b9362c8fbd8 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.258204 28654 ts_tablet_manager.cc:1888] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.258258 28654 log.cc:1196] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/4ca02d87dfd549e9bf01c7f3d38928e3
I20240503 02:09:18.258435 28655 tablet_bootstrap.cc:492] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.258455 28654 ts_tablet_manager.cc:1909] T 4ca02d87dfd549e9bf01c7f3d38928e3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.258478 28655 ts_tablet_manager.cc:1367] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.000s
I20240503 02:09:18.258630 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd59c2a068642c6af4742c990176172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.258610 28655 raft_consensus.cc:348] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.258661 28655 raft_consensus.cc:374] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.258675 28655 raft_consensus.cc:729] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.258702 28656 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.258713 28655 consensus_queue.cc:260] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.258738 28656 ts_tablet_manager.cc:1367] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:18.258792 28655 ts_tablet_manager.cc:1398] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.258865 28655 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.258890 28656 raft_consensus.cc:348] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.258944 28656 raft_consensus.cc:374] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.258960 28656 raft_consensus.cc:729] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.258991 28656 consensus_queue.cc:260] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.259073 28656 ts_tablet_manager.cc:1398] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.259136 28656 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.259182 28655 tablet_bootstrap.cc:654] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.259423 28661 ts_tablet_manager.cc:1888] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240503 02:09:18.259465 28661 log.cc:1196] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/2081f9c133424955a57f3b9362c8fbd8
I20240503 02:09:18.259493 28656 tablet_bootstrap.cc:654] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.259649 28661 ts_tablet_manager.cc:1909] T 2081f9c133424955a57f3b9362c8fbd8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.259855 28664 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.259888 28664 ts_tablet_manager.cc:1367] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:18.260010 28664 raft_consensus.cc:348] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.260053 28664 raft_consensus.cc:374] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.260069 28664 raft_consensus.cc:729] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.260104 28664 consensus_queue.cc:260] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.260185 28664 ts_tablet_manager.cc:1398] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.260264 28664 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.260603 28664 tablet_bootstrap.cc:654] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.260659 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 4ca02d87dfd549e9bf01c7f3d38928e3 (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.260785 28654 tablet_replica.cc:330] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.260850 28654 raft_consensus.cc:2229] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.260885 28654 raft_consensus.cc:2258] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.261497 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
W20240503 02:09:18.261585 27200 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1a6aabb9dc28424088f42567396efdd7 consensus state has no leader
I20240503 02:09:18.262023 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_a604d0eec20e4037ad895d3802e5341f [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
I20240503 02:09:18.262079 28654 ts_tablet_manager.cc:1875] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.262979 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.263051 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a874a3bdc66345b7b0d4c66ef9a9176c" } modify_external_catalogs: true
I20240503 02:09:18.263104 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a874a3bdc66345b7b0d4c66ef9a9176c" } modify_external_catalogs: true
I20240503 02:09:18.263665 28654 ts_tablet_manager.cc:1888] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240503 02:09:18.263901 28654 log.cc:1196] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/2081f9c133424955a57f3b9362c8fbd8
I20240503 02:09:18.264014 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.264045 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_a604d0eec20e4037ad895d3802e5341f [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
I20240503 02:09:18.264173 28664 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.264212 28664 ts_tablet_manager.cc:1367] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:18.264353 28664 raft_consensus.cc:348] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.264406 28664 raft_consensus.cc:374] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.264425 28664 raft_consensus.cc:729] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.264425 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.264458 28664 consensus_queue.cc:260] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.264533 28654 ts_tablet_manager.cc:1909] T 2081f9c133424955a57f3b9362c8fbd8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.264541 28664 ts_tablet_manager.cc:1398] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.264605 28664 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.264604 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.264842 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
W20240503 02:09:18.264925 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 42e6c708a5fd43c09fb1231ec0a1d17e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42e6c708a5fd43c09fb1231ec0a1d17e already in progress: creating tablet
I20240503 02:09:18.264952 28664 tablet_bootstrap.cc:654] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.264940 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 2081f9c133424955a57f3b9362c8fbd8 (table default.Table_a604d0eec20e4037ad895d3802e5341f [id=53e0fca749ea4dc699450e4503b21c03]) successfully deleted
W20240503 02:09:18.265154 27187 catalog_manager.cc:4810] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 42e6c708a5fd43c09fb1231ec0a1d17e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 42e6c708a5fd43c09fb1231ec0a1d17e already in progress: creating tablet
I20240503 02:09:18.265460 28664 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.265460 28655 tablet_bootstrap.cc:492] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.265502 28664 ts_tablet_manager.cc:1367] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.265502 28655 ts_tablet_manager.cc:1367] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.265655 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.265658 28655 raft_consensus.cc:348] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.265714 28655 raft_consensus.cc:374] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.265736 28655 raft_consensus.cc:729] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.265728 28664 raft_consensus.cc:348] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.265794 28664 raft_consensus.cc:374] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.265816 28664 raft_consensus.cc:729] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.265801 28655 consensus_queue.cc:260] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.265857 28664 consensus_queue.cc:260] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.265900 28655 ts_tablet_manager.cc:1398] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.265946 28664 ts_tablet_manager.cc:1398] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.265954 28655 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.266435 28655 tablet_bootstrap.cc:654] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.266494 28656 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.266523 28656 ts_tablet_manager.cc:1367] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.266649 28656 raft_consensus.cc:348] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.266718 28656 raft_consensus.cc:374] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.266737 28656 raft_consensus.cc:729] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.266770 28656 consensus_queue.cc:260] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.266837 28664 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.266866 28656 ts_tablet_manager.cc:1398] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.266911 28656 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.266939 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.267210 28656 tablet_bootstrap.cc:654] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.267211 28664 tablet_bootstrap.cc:654] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.267832 28664 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.267867 28664 ts_tablet_manager.cc:1367] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.267987 28664 raft_consensus.cc:348] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.268038 28664 raft_consensus.cc:374] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.268056 28664 raft_consensus.cc:729] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.268080 28664 consensus_queue.cc:260] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.268154 28664 ts_tablet_manager.cc:1398] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.268739 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.268777 28664 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.268858 28661 tablet_replica.cc:330] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.268913 28661 raft_consensus.cc:2229] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.268949 28661 raft_consensus.cc:2258] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.269147 28661 ts_tablet_manager.cc:1875] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.269150 28664 tablet_bootstrap.cc:654] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.270143 28656 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.270210 28656 ts_tablet_manager.cc:1367] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:18.270346 28656 raft_consensus.cc:348] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.270395 28656 raft_consensus.cc:374] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.270429 28656 raft_consensus.cc:729] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.270464 28656 consensus_queue.cc:260] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.270553 28656 ts_tablet_manager.cc:1398] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.270598 28656 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.270910 28656 tablet_bootstrap.cc:654] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.271597 28656 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.271639 28656 ts_tablet_manager.cc:1367] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.271747 28656 raft_consensus.cc:348] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.271813 28656 raft_consensus.cc:374] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.271831 28656 raft_consensus.cc:729] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.271864 28656 consensus_queue.cc:260] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.271855 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_376b6f6d0e6e4bb68cbe9efbf4608e41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.271945 28656 ts_tablet_manager.cc:1398] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:18.271972 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_376b6f6d0e6e4bb68cbe9efbf4608e41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.271996 28656 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.272420 28656 tablet_bootstrap.cc:654] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.273121 28656 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.273161 28656 ts_tablet_manager.cc:1367] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.273278 28656 raft_consensus.cc:348] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.273386 28656 raft_consensus.cc:374] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.273407 28656 raft_consensus.cc:729] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.273442 28656 consensus_queue.cc:260] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.273553 28656 ts_tablet_manager.cc:1398] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.273900 28664 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.273950 28664 ts_tablet_manager.cc:1367] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:18.274072 28664 raft_consensus.cc:348] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.274127 28664 raft_consensus.cc:374] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.274145 28664 raft_consensus.cc:729] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.274129 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.274179 28664 consensus_queue.cc:260] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.274259 28664 ts_tablet_manager.cc:1398] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.275005 28654 tablet_replica.cc:330] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.275068 28654 raft_consensus.cc:2229] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.275099 28654 raft_consensus.cc:2258] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.275636 28654 ts_tablet_manager.cc:1875] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.275897 28661 ts_tablet_manager.cc:1888] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.275944 28661 log.cc:1196] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.276106 28661 ts_tablet_manager.cc:1909] T 42e6c708a5fd43c09fb1231ec0a1d17e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.276262 29551 raft_consensus.cc:482] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.276330 29551 raft_consensus.cc:504] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.276445 29551 leader_election.cc:288] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.276628 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 42e6c708a5fd43c09fb1231ec0a1d17e (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.276700 28655 tablet_bootstrap.cc:492] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.276741 28655 ts_tablet_manager.cc:1367] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:18.276712 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6aabb9dc28424088f42567396efdd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.276772 27719 raft_consensus.cc:2454] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.276929 27388 leader_election.cc:302] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.276991 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6aabb9dc28424088f42567396efdd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.277020 29551 raft_consensus.cc:2781] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.277052 29551 raft_consensus.cc:482] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.277048 27586 raft_consensus.cc:2454] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.277071 29551 raft_consensus.cc:3037] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.277045 28654 ts_tablet_manager.cc:1888] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.277151 28654 log.cc:1196] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.277351 28654 ts_tablet_manager.cc:1909] T 42e6c708a5fd43c09fb1231ec0a1d17e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.277762 29551 raft_consensus.cc:504] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.277874 29551 leader_election.cc:288] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.278002 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6aabb9dc28424088f42567396efdd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.278064 27719 raft_consensus.cc:3037] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.276908 28655 raft_consensus.cc:348] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.278211 28655 raft_consensus.cc:374] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.278244 28655 raft_consensus.cc:729] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.278457 28655 consensus_queue.cc:260] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.278582 28655 ts_tablet_manager.cc:1398] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.278653 27719 raft_consensus.cc:2454] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.278767 28655 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.278831 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a6aabb9dc28424088f42567396efdd7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.278887 27586 raft_consensus.cc:3037] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.278877 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 42e6c708a5fd43c09fb1231ec0a1d17e (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.278959 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9dfb0c37a6524daf9700c7ee14eb796f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.279052 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9dfb0c37a6524daf9700c7ee14eb796f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.279245 28655 tablet_bootstrap.cc:654] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.279263 27560 tablet_service.cc:1467] Processing CreateTablet for tablet e6bb29693c414ac98223c93f543ad14f (DEFAULT_TABLE table=default.Table_376b6f6d0e6e4bb68cbe9efbf4608e41 [id=39d3892933fd4225a05cb75492fbb2b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.279368 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bb29693c414ac98223c93f543ad14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.279441 27388 leader_election.cc:302] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.279474 27586 raft_consensus.cc:2454] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.279685 28774 consensus_queue.cc:1035] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.279810 29551 raft_consensus.cc:2781] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.279866 29551 raft_consensus.cc:686] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.279920 29551 consensus_queue.cc:237] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.280256 28655 tablet_bootstrap.cc:492] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.280297 28655 ts_tablet_manager.cc:1367] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.280427 28655 raft_consensus.cc:348] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.280474 28655 raft_consensus.cc:374] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.280489 28655 raft_consensus.cc:729] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.280521 28655 consensus_queue.cc:260] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.280611 28655 ts_tablet_manager.cc:1398] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.280653 28655 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.280974 28655 tablet_bootstrap.cc:654] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.281031 27702 tablet_service.cc:1467] Processing CreateTablet for tablet e6bb29693c414ac98223c93f543ad14f (DEFAULT_TABLE table=default.Table_376b6f6d0e6e4bb68cbe9efbf4608e41 [id=39d3892933fd4225a05cb75492fbb2b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.281114 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bb29693c414ac98223c93f543ad14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.281417 27422 tablet_service.cc:1467] Processing CreateTablet for tablet e6bb29693c414ac98223c93f543ad14f (DEFAULT_TABLE table=default.Table_376b6f6d0e6e4bb68cbe9efbf4608e41 [id=39d3892933fd4225a05cb75492fbb2b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.281503 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6bb29693c414ac98223c93f543ad14f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.281302 28656 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.282357 28656 tablet_bootstrap.cc:654] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.283282 28693 consensus_queue.cc:1035] T 99265e78c1d74facb95bb8f7abf71110 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.283591 28664 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.283941 28664 tablet_bootstrap.cc:654] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.284513 28507 raft_consensus.cc:482] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.284569 28507 raft_consensus.cc:504] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.284675 28507 leader_election.cc:288] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.284857 28664 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.284904 28664 ts_tablet_manager.cc:1367] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.285018 28664 raft_consensus.cc:348] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:18.285043 27523 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
I20240503 02:09:18.285076 28664 raft_consensus.cc:374] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.285094 28664 raft_consensus.cc:729] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.285135 28664 consensus_queue.cc:260] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.285183 28507 consensus_queue.cc:1035] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.285230 28664 ts_tablet_manager.cc:1398] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.285686 28655 tablet_bootstrap.cc:492] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.285727 28655 ts_tablet_manager.cc:1367] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.285838 28655 raft_consensus.cc:348] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.285893 28655 raft_consensus.cc:374] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.285910 28655 raft_consensus.cc:729] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
W20240503 02:09:18.285936 27524 leader_election.cc:341] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 38d89374090345758db303c33b460c9a
I20240503 02:09:18.285951 28655 consensus_queue.cc:260] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.285995 27524 leader_election.cc:302] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.286047 28655 ts_tablet_manager.cc:1398] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.286098 28655 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.286249 28507 raft_consensus.cc:2726] T 38d89374090345758db303c33b460c9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:18.286419 28655 tablet_bootstrap.cc:654] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.286396 27198 catalog_manager.cc:5484] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.286491 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.286628 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 (DEFAULT_TABLE table=default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.287005 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 (DEFAULT_TABLE table=default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.287097 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3faa2ff595844bd3a1aa8636c70a64b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.287103 28655 tablet_bootstrap.cc:492] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.287139 28655 ts_tablet_manager.cc:1367] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.287345 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3faa2ff595844bd3a1aa8636c70a64b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.287338 28655 raft_consensus.cc:348] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.287395 28655 raft_consensus.cc:374] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.287416 28655 raft_consensus.cc:729] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.287459 28655 consensus_queue.cc:260] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.287569 28655 ts_tablet_manager.cc:1398] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.287637 28655 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.287813 28656 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.287853 28656 ts_tablet_manager.cc:1367] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.287967 28656 raft_consensus.cc:348] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.287987 28655 tablet_bootstrap.cc:654] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.288017 28656 raft_consensus.cc:374] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.288036 28656 raft_consensus.cc:729] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.288071 28656 consensus_queue.cc:260] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.288153 28656 ts_tablet_manager.cc:1398] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.288455 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 (DEFAULT_TABLE table=default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.288537 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3faa2ff595844bd3a1aa8636c70a64b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.288564 28656 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.288920 28656 tablet_bootstrap.cc:654] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.289474 28656 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.289518 28656 ts_tablet_manager.cc:1367] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.289647 28656 raft_consensus.cc:348] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.289706 28656 raft_consensus.cc:374] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.289726 28656 raft_consensus.cc:729] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.289764 28656 consensus_queue.cc:260] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.289882 28656 ts_tablet_manager.cc:1398] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.289877 29551 raft_consensus.cc:482] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.289921 29551 raft_consensus.cc:504] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.290004 29551 leader_election.cc:288] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.290208 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540e09a284246ee92301479690ed27c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.290277 27586 raft_consensus.cc:2454] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.290313 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540e09a284246ee92301479690ed27c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.290385 27718 raft_consensus.cc:2454] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.290556 28664 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.290664 27388 leader_election.cc:302] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.290867 28767 raft_consensus.cc:2781] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.290904 28664 tablet_bootstrap.cc:654] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.290913 28767 raft_consensus.cc:482] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.290937 28767 raft_consensus.cc:3037] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.291301 28507 consensus_queue.cc:1035] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.291476 28767 raft_consensus.cc:504] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.291570 28767 leader_election.cc:288] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.291899 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540e09a284246ee92301479690ed27c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.291963 27586 raft_consensus.cc:3037] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.291986 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540e09a284246ee92301479690ed27c" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.292043 27718 raft_consensus.cc:3037] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.292557 27586 raft_consensus.cc:2454] T 6540e09a284246ee92301479690ed27c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.292641 27718 raft_consensus.cc:2454] T 6540e09a284246ee92301479690ed27c P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.292711 27386 leader_election.cc:302] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.292768 29551 raft_consensus.cc:2781] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.292794 29551 raft_consensus.cc:686] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.292824 29551 consensus_queue.cc:237] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.294191 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
W20240503 02:09:18.294378 27187 catalog_manager.cc:4797] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): delete failed for tablet 42e6c708a5fd43c09fb1231ec0a1d17e because the tablet was not found. No further retry: Not found: Tablet not found: 42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.294697 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e6c708a5fd43c09fb1231ec0a1d17e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.294772 28653 tablet_replica.cc:330] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.294819 28653 raft_consensus.cc:2229] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.294858 28653 raft_consensus.cc:2258] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.294936 28655 tablet_bootstrap.cc:492] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.294976 28655 ts_tablet_manager.cc:1367] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.295014 28653 ts_tablet_manager.cc:1875] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.295114 28655 raft_consensus.cc:348] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.295169 28655 raft_consensus.cc:374] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.295189 28655 raft_consensus.cc:729] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.295225 28655 consensus_queue.cc:260] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.295323 28655 ts_tablet_manager.cc:1398] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.295393 28655 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.295753 28655 tablet_bootstrap.cc:654] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.296025 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5de925a9e48a44d496c1eba7e7609f94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.296129 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5de925a9e48a44d496c1eba7e7609f94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.296402 28664 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.296344 27198 catalog_manager.cc:5484] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.296433 28653 ts_tablet_manager.cc:1888] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.296442 28664 ts_tablet_manager.cc:1367] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:18.296475 28653 log.cc:1196] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/42e6c708a5fd43c09fb1231ec0a1d17e
I20240503 02:09:18.296572 28664 raft_consensus.cc:348] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.296623 28664 raft_consensus.cc:374] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.296640 28664 raft_consensus.cc:729] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.296661 28653 ts_tablet_manager.cc:1909] T 42e6c708a5fd43c09fb1231ec0a1d17e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.296674 28664 consensus_queue.cc:260] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.296778 28664 ts_tablet_manager.cc:1398] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.297055 28655 tablet_bootstrap.cc:492] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.297097 28655 ts_tablet_manager.cc:1367] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.297232 28655 raft_consensus.cc:348] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.297299 28655 raft_consensus.cc:374] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.297348 28655 raft_consensus.cc:729] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.297390 28655 consensus_queue.cc:260] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.297495 28655 ts_tablet_manager.cc:1398] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.299182 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_872bf15f57494be09618084179db15e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.299286 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_872bf15f57494be09618084179db15e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.300643 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 42e6c708a5fd43c09fb1231ec0a1d17e (table default.Table_a874a3bdc66345b7b0d4c66ef9a9176c [id=b7486d39439f46fc945e04fb12c4d5aa]) successfully deleted
I20240503 02:09:18.301430 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 941c3021996a4962b87b068018948e67 (DEFAULT_TABLE table=default.Table_5de925a9e48a44d496c1eba7e7609f94 [id=54b500a388f247e5a660071a6c5548e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.301533 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941c3021996a4962b87b068018948e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.301717 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 941c3021996a4962b87b068018948e67 (DEFAULT_TABLE table=default.Table_5de925a9e48a44d496c1eba7e7609f94 [id=54b500a388f247e5a660071a6c5548e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.301796 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941c3021996a4962b87b068018948e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.302621 28664 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.302955 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b37202fb404544feb7a43e4f948b9eb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.303057 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b37202fb404544feb7a43e4f948b9eb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.303062 28664 tablet_bootstrap.cc:654] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.303582 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 941c3021996a4962b87b068018948e67 (DEFAULT_TABLE table=default.Table_5de925a9e48a44d496c1eba7e7609f94 [id=54b500a388f247e5a660071a6c5548e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.303675 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 941c3021996a4962b87b068018948e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.303793 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 4831e4a404b34b88b0511d2e5eca0897 (DEFAULT_TABLE table=default.Table_872bf15f57494be09618084179db15e2 [id=fd9156bab3034c2aac452921948128e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.303865 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4831e4a404b34b88b0511d2e5eca0897. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.304111 28656 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.304482 28656 tablet_bootstrap.cc:654] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.303682 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 4831e4a404b34b88b0511d2e5eca0897 (DEFAULT_TABLE table=default.Table_872bf15f57494be09618084179db15e2 [id=fd9156bab3034c2aac452921948128e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.304790 28655 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.304867 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4831e4a404b34b88b0511d2e5eca0897. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.305089 28655 tablet_bootstrap.cc:654] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.305558 28655 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.305600 28655 ts_tablet_manager.cc:1367] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.305723 28655 raft_consensus.cc:348] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.305783 28655 raft_consensus.cc:374] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.305804 28655 raft_consensus.cc:729] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.305845 28655 consensus_queue.cc:260] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.305939 28655 ts_tablet_manager.cc:1398] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.306447 28664 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.306486 28664 ts_tablet_manager.cc:1367] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:18.306609 28664 raft_consensus.cc:348] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.306663 28664 raft_consensus.cc:374] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.306680 28664 raft_consensus.cc:729] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.306715 28664 consensus_queue.cc:260] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.306814 28664 ts_tablet_manager.cc:1398] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.307132 28767 consensus_queue.cc:1035] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.307143 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_1db3284882814b5a817c185b2ba87962" } new_table_name: "default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e" modify_external_catalogs: true
I20240503 02:09:18.307641 27719 raft_consensus.cc:1261] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.307716 27560 tablet_service.cc:1467] Processing CreateTablet for tablet ea0e3d17da1d4d189ce0fdbf5725e289 (DEFAULT_TABLE table=default.Table_b37202fb404544feb7a43e4f948b9eb8 [id=b3d8c859fddf4538b03907363b4d84b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.307796 27586 raft_consensus.cc:1261] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.307898 27703 tablet_service.cc:1467] Processing CreateTablet for tablet ea0e3d17da1d4d189ce0fdbf5725e289 (DEFAULT_TABLE table=default.Table_b37202fb404544feb7a43e4f948b9eb8 [id=b3d8c859fddf4538b03907363b4d84b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.307902 28767 consensus_queue.cc:1035] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.307982 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0e3d17da1d4d189ce0fdbf5725e289. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.307803 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0e3d17da1d4d189ce0fdbf5725e289. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.308053 28767 consensus_queue.cc:1035] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.309177 28664 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.309372 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 4831e4a404b34b88b0511d2e5eca0897 (DEFAULT_TABLE table=default.Table_872bf15f57494be09618084179db15e2 [id=fd9156bab3034c2aac452921948128e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.309458 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4831e4a404b34b88b0511d2e5eca0897. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.309527 28664 tablet_bootstrap.cc:654] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.310340 28655 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.310771 28655 tablet_bootstrap.cc:654] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.311316 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.311715 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 1a6aabb9dc28424088f42567396efdd7 (table default.Table_6f1cfab914c2494a97e80c38bef7a4ab [id=b295d0ea3d4a46138674b9ca6d000698]). Expected version 1 got 0
I20240503 02:09:18.311939 27422 tablet_service.cc:1467] Processing CreateTablet for tablet ea0e3d17da1d4d189ce0fdbf5725e289 (DEFAULT_TABLE table=default.Table_b37202fb404544feb7a43e4f948b9eb8 [id=b3d8c859fddf4538b03907363b4d84b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.312034 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea0e3d17da1d4d189ce0fdbf5725e289. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.313405 28693 consensus_queue.cc:1035] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.313602 29551 consensus_queue.cc:1035] T 7078864d79114bbe910112efb8ca01c3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.313900 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9fd323db7d3144798a96ca57ddaaafe2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.313994 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fd323db7d3144798a96ca57ddaaafe2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.315675 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_254bc1bf83b3425fa3fbc0a902895bdc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.315766 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_254bc1bf83b3425fa3fbc0a902895bdc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.317939 28664 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.317983 28664 ts_tablet_manager.cc:1367] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:18.318082 28656 tablet_bootstrap.cc:492] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.318116 28656 ts_tablet_manager.cc:1367] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:18.318117 28664 raft_consensus.cc:348] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.318171 28664 raft_consensus.cc:374] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.318188 28664 raft_consensus.cc:729] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.318223 28664 consensus_queue.cc:260] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.318248 28656 raft_consensus.cc:348] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.318295 28656 raft_consensus.cc:374] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.318310 28656 raft_consensus.cc:729] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.318313 28664 ts_tablet_manager.cc:1398] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.318361 28664 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.318356 28656 consensus_queue.cc:260] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.318439 28656 ts_tablet_manager.cc:1398] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.318502 28656 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.318699 28664 tablet_bootstrap.cc:654] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.318810 28656 tablet_bootstrap.cc:654] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.319458 28664 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.319497 28664 ts_tablet_manager.cc:1367] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.319586 27560 tablet_service.cc:1467] Processing CreateTablet for tablet b60278e574634b7aaba6ef4412dd2dba (DEFAULT_TABLE table=default.Table_254bc1bf83b3425fa3fbc0a902895bdc [id=26cf7e679f744b47b2e991bc7e085b4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.319623 28664 raft_consensus.cc:348] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.319689 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60278e574634b7aaba6ef4412dd2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.319694 28664 raft_consensus.cc:374] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.319715 28664 raft_consensus.cc:729] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.319748 28664 consensus_queue.cc:260] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.319839 28664 ts_tablet_manager.cc:1398] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.320710 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_37f488e04c30428180cb81f0d77e2ac0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.320814 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37f488e04c30428180cb81f0d77e2ac0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.321192 28693 consensus_queue.cc:1035] T 9001e72388f7452e85476a5d2b3cf1c1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.322402 30084 mvcc.cc:205] Tried to move back new op lower bound from 7023420040426594304 to 7023420040314781696. Current Snapshot: MvccSnapshot[applied={T|T < 7023420040426594304}]
I20240503 02:09:18.322471 30190 tablet.cc:1722] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.322480 28685 raft_consensus.cc:482] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.322528 28685 raft_consensus.cc:504] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.322568 30170 tablet.cc:1722] T 1a6aabb9dc28424088f42567396efdd7 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.322623 28685 leader_election.cc:288] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.322934 30097 tablet.cc:1722] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.322988 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e76ee4b127b944ee8611ac188dbcc087" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.323048 27585 raft_consensus.cc:2454] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.323082 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e76ee4b127b944ee8611ac188dbcc087" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.323132 27722 raft_consensus.cc:2454] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.323343 27388 leader_election.cc:302] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.323411 28685 raft_consensus.cc:2781] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.323431 28685 raft_consensus.cc:482] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.323422 29558 tablet.cc:1722] T 1a6aabb9dc28424088f42567396efdd7 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.323451 28685 raft_consensus.cc:3037] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.323869 28656 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.323925 28656 ts_tablet_manager.cc:1367] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.324045 28656 raft_consensus.cc:348] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.324093 28656 raft_consensus.cc:374] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.324111 28656 raft_consensus.cc:729] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.324152 28656 consensus_queue.cc:260] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.324139 28685 raft_consensus.cc:504] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.324232 28685 leader_election.cc:288] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.324251 28656 ts_tablet_manager.cc:1398] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.324306 28656 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.324498 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e76ee4b127b944ee8611ac188dbcc087" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.324556 27722 raft_consensus.cc:3037] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.324697 28656 tablet_bootstrap.cc:654] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.324676 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e76ee4b127b944ee8611ac188dbcc087" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.324734 27585 raft_consensus.cc:3037] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.325245 27722 raft_consensus.cc:2454] T e76ee4b127b944ee8611ac188dbcc087 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.325305 27585 raft_consensus.cc:2454] T e76ee4b127b944ee8611ac188dbcc087 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.325475 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 7b820e0b556240b393ee31f0eae3a36b (DEFAULT_TABLE table=default.Table_9fd323db7d3144798a96ca57ddaaafe2 [id=988d19150c7148f19648ef6a7eea345b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.325522 27388 leader_election.cc:302] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.325558 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b820e0b556240b393ee31f0eae3a36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.325618 28685 raft_consensus.cc:2781] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.325661 28685 raft_consensus.cc:686] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.325695 28685 consensus_queue.cc:237] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.326380 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 7b820e0b556240b393ee31f0eae3a36b (DEFAULT_TABLE table=default.Table_9fd323db7d3144798a96ca57ddaaafe2 [id=988d19150c7148f19648ef6a7eea345b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.326470 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b820e0b556240b393ee31f0eae3a36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.326640 28710 tablet.cc:1722] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.326913 28664 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.327246 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 40e6a23f6efb49b098699bc6a91e1709 (DEFAULT_TABLE table=default.Table_37f488e04c30428180cb81f0d77e2ac0 [id=35e3ed9194c44c748fe29886992b08cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.327298 28664 tablet_bootstrap.cc:654] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.327330 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e6a23f6efb49b098699bc6a91e1709. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.327430 27195 catalog_manager.cc:5484] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.327704 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_f42523eb93df4875ac58a242ed2d93a5" } new_table_name: "default.Table_135e297a3dbc415f82c9a712a83d80d3" modify_external_catalogs: true
I20240503 02:09:18.327860 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3db8837099ba4da6aedfe7bb78c2b64a" } modify_external_catalogs: true
I20240503 02:09:18.327906 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3db8837099ba4da6aedfe7bb78c2b64a" } modify_external_catalogs: true
I20240503 02:09:18.328976 27848 raft_consensus.cc:482] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.329025 27848 raft_consensus.cc:504] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.329104 27848 leader_election.cc:288] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.329298 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 40e6a23f6efb49b098699bc6a91e1709 (DEFAULT_TABLE table=default.Table_37f488e04c30428180cb81f0d77e2ac0 [id=35e3ed9194c44c748fe29886992b08cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.329422 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e6a23f6efb49b098699bc6a91e1709. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.330448 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet be954e1ee239497fad20d030a835e2c1
I20240503 02:09:18.330889 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97721d6cf071459baaa04b58a36c7013" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.330960 27447 raft_consensus.cc:2454] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.331454 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet be954e1ee239497fad20d030a835e2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.331490 27524 leader_election.cc:302] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.331610 27447 raft_consensus.cc:3037] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.331749 28654 tablet_replica.cc:330] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.331821 28654 raft_consensus.cc:2229] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.331877 28654 raft_consensus.cc:2258] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.332042 29241 raft_consensus.cc:2781] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.332072 29241 raft_consensus.cc:482] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.332095 29241 raft_consensus.cc:3037] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.332247 27447 raft_consensus.cc:1261] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.332435 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet be954e1ee239497fad20d030a835e2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.332640 29241 raft_consensus.cc:504] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.332736 29241 leader_election.cc:288] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.332767 28507 consensus_queue.cc:1035] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.332868 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97721d6cf071459baaa04b58a36c7013" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.332916 27447 raft_consensus.cc:3037] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.332055 28654 ts_tablet_manager.cc:1875] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.333434 27447 raft_consensus.cc:2454] T 97721d6cf071459baaa04b58a36c7013 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.333608 28693 raft_consensus.cc:482] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.333658 28693 raft_consensus.cc:504] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.333777 28693 leader_election.cc:288] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.333937 27524 leader_election.cc:302] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.333947 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bb29693c414ac98223c93f543ad14f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.334015 28507 raft_consensus.cc:2781] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.334030 27585 raft_consensus.cc:2454] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.334054 28507 raft_consensus.cc:686] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.334105 28507 consensus_queue.cc:237] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.334278 28654 ts_tablet_manager.cc:1888] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.334326 28654 log.cc:1196] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/be954e1ee239497fad20d030a835e2c1
I20240503 02:09:18.334349 27656 leader_election.cc:302] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.334424 28693 raft_consensus.cc:2781] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.334444 28693 raft_consensus.cc:482] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.334460 28693 raft_consensus.cc:3037] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.334504 28654 ts_tablet_manager.cc:1909] T be954e1ee239497fad20d030a835e2c1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.334612 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bb29693c414ac98223c93f543ad14f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.334662 27453 raft_consensus.cc:2454] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.335052 27719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97721d6cf071459baaa04b58a36c7013" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.335104 28693 raft_consensus.cc:504] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.335181 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97721d6cf071459baaa04b58a36c7013" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.335239 28693 leader_election.cc:288] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.335114 27719 raft_consensus.cc:3037] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.335297 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 7b820e0b556240b393ee31f0eae3a36b (DEFAULT_TABLE table=default.Table_9fd323db7d3144798a96ca57ddaaafe2 [id=988d19150c7148f19648ef6a7eea345b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.335376 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b820e0b556240b393ee31f0eae3a36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.335855 27719 raft_consensus.cc:2454] T 97721d6cf071459baaa04b58a36c7013 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.336081 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bb29693c414ac98223c93f543ad14f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.336107 27718 raft_consensus.cc:1261] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer 5cc801b4e0b047b3b4b30f8dd5c7c5db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.336143 27585 raft_consensus.cc:3037] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.336210 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6bb29693c414ac98223c93f543ad14f" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.336267 27453 raft_consensus.cc:3037] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.336663 28653 tablet_replica.cc:330] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.336716 27585 raft_consensus.cc:2454] T e6bb29693c414ac98223c93f543ad14f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.336722 28653 raft_consensus.cc:2229] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.336761 28653 raft_consensus.cc:2258] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.336828 27453 raft_consensus.cc:2454] T e6bb29693c414ac98223c93f543ad14f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.336943 27656 leader_election.cc:302] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.337074 30009 tablet.cc:1722] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.337080 28740 raft_consensus.cc:2781] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.337136 28740 raft_consensus.cc:686] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.337170 28740 consensus_queue.cc:237] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.337617 28653 ts_tablet_manager.cc:1875] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.338490 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet be954e1ee239497fad20d030a835e2c1 (table default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]) successfully deleted
I20240503 02:09:18.338902 28507 consensus_queue.cc:1035] T 9d08a51420834ca0afffd14e8743e1c5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.339006 28653 ts_tablet_manager.cc:1888] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.339046 28653 log.cc:1196] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/be954e1ee239497fad20d030a835e2c1
I20240503 02:09:18.339213 28653 ts_tablet_manager.cc:1909] T be954e1ee239497fad20d030a835e2c1 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.339192 27195 catalog_manager.cc:5484] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.339506 27197 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9d08a51420834ca0afffd14e8743e1c5 (table default.Table_135e297a3dbc415f82c9a712a83d80d3 [id=a7566949ac244087ae89f85cb0f6fa79]). Expected version 1 got 0
I20240503 02:09:18.339648 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet be954e1ee239497fad20d030a835e2c1 (table default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]) successfully deleted
I20240503 02:09:18.340978 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 40e6a23f6efb49b098699bc6a91e1709 (DEFAULT_TABLE table=default.Table_37f488e04c30428180cb81f0d77e2ac0 [id=35e3ed9194c44c748fe29886992b08cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.341081 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40e6a23f6efb49b098699bc6a91e1709. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.341750 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_5039eaa3c51b49c2a63c812673bcf574"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.341851 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5039eaa3c51b49c2a63c812673bcf574 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.341897 30365 tablet.cc:1722] T 9d08a51420834ca0afffd14e8743e1c5 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.343508 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_135e297a3dbc415f82c9a712a83d80d3 [id=a7566949ac244087ae89f85cb0f6fa79] alter complete (version 1)
I20240503 02:09:18.344664 27195 catalog_manager.cc:5484] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.344765 27195 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9d08a51420834ca0afffd14e8743e1c5 (table default.Table_135e297a3dbc415f82c9a712a83d80d3 [id=a7566949ac244087ae89f85cb0f6fa79]). Expected version 1 got 0
I20240503 02:09:18.344879 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec2373bab15403cbcae47fd20f3c7ce (DEFAULT_TABLE table=default.Table_5039eaa3c51b49c2a63c812673bcf574 [id=4055f458c5e9488b9821b338d847cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.344960 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec2373bab15403cbcae47fd20f3c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.345562 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec2373bab15403cbcae47fd20f3c7ce (DEFAULT_TABLE table=default.Table_5039eaa3c51b49c2a63c812673bcf574 [id=4055f458c5e9488b9821b338d847cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.345671 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec2373bab15403cbcae47fd20f3c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.346148 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec2373bab15403cbcae47fd20f3c7ce (DEFAULT_TABLE table=default.Table_5039eaa3c51b49c2a63c812673bcf574 [id=4055f458c5e9488b9821b338d847cbaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.346228 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec2373bab15403cbcae47fd20f3c7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.346486 27702 tablet_service.cc:1467] Processing CreateTablet for tablet b60278e574634b7aaba6ef4412dd2dba (DEFAULT_TABLE table=default.Table_254bc1bf83b3425fa3fbc0a902895bdc [id=26cf7e679f744b47b2e991bc7e085b4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.346609 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60278e574634b7aaba6ef4412dd2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.346495 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet be954e1ee239497fad20d030a835e2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.348055 28661 tablet_replica.cc:330] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.348109 28661 raft_consensus.cc:2229] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.348138 28661 raft_consensus.cc:2258] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.348331 28661 ts_tablet_manager.cc:1875] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.348897 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bb6cee84b5624c2c93af79f09ea9d32e" } new_table_name: "default.Table_4cbe6ae4d7044edcb3daf09038239113" modify_external_catalogs: true
I20240503 02:09:18.349469 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_405719b600a24ad69a35850163c91c9c" } new_table_name: "default.Table_8af617eea38240058fed7fa3a092a853" modify_external_catalogs: true
I20240503 02:09:18.349570 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f] alter complete (version 1)
I20240503 02:09:18.349756 28661 ts_tablet_manager.cc:1888] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.349789 28661 log.cc:1196] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/be954e1ee239497fad20d030a835e2c1
I20240503 02:09:18.349987 28661 ts_tablet_manager.cc:1909] T be954e1ee239497fad20d030a835e2c1 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.350395 27422 tablet_service.cc:1467] Processing CreateTablet for tablet b60278e574634b7aaba6ef4412dd2dba (DEFAULT_TABLE table=default.Table_254bc1bf83b3425fa3fbc0a902895bdc [id=26cf7e679f744b47b2e991bc7e085b4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.350530 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60278e574634b7aaba6ef4412dd2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.350531 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet be954e1ee239497fad20d030a835e2c1 (table default.Table_3db8837099ba4da6aedfe7bb78c2b64a [id=4d61d0f0d5a34238b98a3387854b52a8]) successfully deleted
I20240503 02:09:18.352116 30190 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.352449 27718 raft_consensus.cc:1261] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.352669 30364 tablet.cc:1722] T 9d08a51420834ca0afffd14e8743e1c5 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.352947 27587 raft_consensus.cc:3037] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.353173 28656 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.353209 28656 ts_tablet_manager.cc:1367] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240503 02:09:18.353369 28656 raft_consensus.cc:348] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.353422 28656 raft_consensus.cc:374] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.353440 28656 raft_consensus.cc:729] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.353467 28656 consensus_queue.cc:260] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.353561 28656 ts_tablet_manager.cc:1398] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.353575 27587 raft_consensus.cc:1261] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.353741 28685 consensus_queue.cc:1035] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.354336 28656 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.354599 28685 consensus_queue.cc:1035] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.354691 28656 tablet_bootstrap.cc:654] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.355192 28656 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.355233 28656 ts_tablet_manager.cc:1367] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.355345 28656 raft_consensus.cc:348] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.355387 28656 raft_consensus.cc:374] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.355402 28656 raft_consensus.cc:729] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.355445 28656 consensus_queue.cc:260] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.355551 28656 ts_tablet_manager.cc:1398] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.355597 28656 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.355963 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3 (table default.Table_4cbe6ae4d7044edcb3daf09038239113 [id=a26f902c47ff4389af95416eadbddfd5]). Expected version 1 got 0
I20240503 02:09:18.356010 28656 tablet_bootstrap.cc:654] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.357041 30364 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.357159 30170 tablet.cc:1722] T a24e57083d8f4dd9b450b0c78a611e0f P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.357882 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet b24a6595e7ea4b13a2cb7e5894bdc5d3 (table default.Table_4cbe6ae4d7044edcb3daf09038239113 [id=a26f902c47ff4389af95416eadbddfd5]). Expected version 1 got 0
I20240503 02:09:18.358484 28664 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.358527 28664 ts_tablet_manager.cc:1367] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.001s
I20240503 02:09:18.358675 28664 raft_consensus.cc:348] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.358727 28664 raft_consensus.cc:374] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.358757 28664 raft_consensus.cc:729] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.358793 28664 consensus_queue.cc:260] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.358884 28664 ts_tablet_manager.cc:1398] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.358930 28664 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.359256 28664 tablet_bootstrap.cc:654] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.360070 28693 raft_consensus.cc:482] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.360117 28693 raft_consensus.cc:504] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.360198 28693 leader_election.cc:288] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.360404 28685 raft_consensus.cc:482] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.360427 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.360457 28685 raft_consensus.cc:504] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.360538 28685 leader_election.cc:288] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
W20240503 02:09:18.360612 27659 leader_election.cc:341] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.360644 28664 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.360690 28664 ts_tablet_manager.cc:1367] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.360814 28664 raft_consensus.cc:348] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.360863 28664 raft_consensus.cc:374] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.360880 28664 raft_consensus.cc:729] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.360855 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c869e9a2d9a4472a65d57af51b491a1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.360908 27587 raft_consensus.cc:2454] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.360904 28664 consensus_queue.cc:260] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.360985 28664 ts_tablet_manager.cc:1398] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.361034 28664 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.361392 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c869e9a2d9a4472a65d57af51b491a1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.361404 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.361454 27585 raft_consensus.cc:2454] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.361454 27718 raft_consensus.cc:2454] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.361531 27386 leader_election.cc:302] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.361593 28766 raft_consensus.cc:2781] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.361611 30170 tablet.cc:1722] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.361613 28766 raft_consensus.cc:482] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.361725 28766 raft_consensus.cc:3037] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.361905 28664 tablet_bootstrap.cc:654] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.362051 27656 leader_election.cc:302] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.362154 28693 raft_consensus.cc:2781] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.362180 28693 raft_consensus.cc:482] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.362197 28693 raft_consensus.cc:3037] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.362339 28766 raft_consensus.cc:504] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.362432 28766 leader_election.cc:288] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.362716 28664 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.362761 28664 ts_tablet_manager.cc:1367] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.362813 28693 raft_consensus.cc:504] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.362859 28766 raft_consensus.cc:482] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.362895 28766 raft_consensus.cc:504] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.362927 28664 raft_consensus.cc:348] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.362984 28664 raft_consensus.cc:374] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.362985 28766 leader_election.cc:288] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.363003 28664 raft_consensus.cc:729] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.362980 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c869e9a2d9a4472a65d57af51b491a1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.363026 27585 raft_consensus.cc:3037] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.363039 28664 consensus_queue.cc:260] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.363134 28664 ts_tablet_manager.cc:1398] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.363194 28664 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.363301 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c869e9a2d9a4472a65d57af51b491a1" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.363358 27722 raft_consensus.cc:3037] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.363516 28664 tablet_bootstrap.cc:654] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.363638 27585 raft_consensus.cc:2454] T 6c869e9a2d9a4472a65d57af51b491a1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.363723 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.363806 27720 raft_consensus.cc:2454] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.363831 30364 tablet.cc:1722] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.363945 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.363970 27722 raft_consensus.cc:2454] T 6c869e9a2d9a4472a65d57af51b491a1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.363984 27388 leader_election.cc:302] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.363994 27587 raft_consensus.cc:3037] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.364116 28766 raft_consensus.cc:2781] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.364146 28766 raft_consensus.cc:482] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.364162 28766 raft_consensus.cc:3037] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.364614 27587 raft_consensus.cc:2454] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.364715 28766 raft_consensus.cc:504] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.364742 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.364791 27584 raft_consensus.cc:2454] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.364807 28766 leader_election.cc:288] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.364984 27656 leader_election.cc:302] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.365075 27853 raft_consensus.cc:2781] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.365109 27853 raft_consensus.cc:686] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.365118 27388 leader_election.cc:302] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.365145 27853 consensus_queue.cc:237] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.365172 28766 raft_consensus.cc:2781] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.365200 28766 raft_consensus.cc:686] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.365234 28766 consensus_queue.cc:237] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.365252 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.365304 27718 raft_consensus.cc:3037] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.362931 28693 leader_election.cc:288] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.365893 27718 raft_consensus.cc:2454] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.365922 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_8af617eea38240058fed7fa3a092a853 [id=09f75c45684343a9bae908f60652e20e] alter complete (version 3)
I20240503 02:09:18.366183 28693 consensus_queue.cc:1035] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.366298 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.366360 27388 leader_election.cc:302] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.366425 28685 raft_consensus.cc:2781] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.366466 28685 raft_consensus.cc:686] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.366498 28685 consensus_queue.cc:237] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
W20240503 02:09:18.366511 27659 leader_election.cc:341] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.366798 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c9984b7488438a863dcf791b5b794f" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.366863 27587 raft_consensus.cc:3037] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.367439 27587 raft_consensus.cc:2454] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.368659 27198 catalog_manager.cc:5484] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.368767 27198 catalog_manager.cc:5484] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.369869 28664 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.369912 28664 ts_tablet_manager.cc:1367] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:18.370031 28664 raft_consensus.cc:348] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.370081 28664 raft_consensus.cc:374] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.370097 28664 raft_consensus.cc:729] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.370128 28664 consensus_queue.cc:260] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.370164 29643 consensus_queue.cc:1035] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.370203 28664 ts_tablet_manager.cc:1398] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.370301 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9fd323db7d3144798a96ca57ddaaafe2" } new_table_name: "default.Table_2248635d13b248b3a0d8a63ce5e8a324" modify_external_catalogs: true
I20240503 02:09:18.372685 27200 catalog_manager.cc:5484] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.372807 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_4cbe6ae4d7044edcb3daf09038239113 [id=a26f902c47ff4389af95416eadbddfd5] alter complete (version 1)
W20240503 02:09:18.373792 27195 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b820e0b556240b393ee31f0eae3a36b consensus state has no leader
I20240503 02:09:18.375388 30190 tablet.cc:1722] T 1a6aabb9dc28424088f42567396efdd7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.377508 28655 tablet_bootstrap.cc:492] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.377550 28655 ts_tablet_manager.cc:1367] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.001s
I20240503 02:09:18.377648 28655 raft_consensus.cc:348] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.377682 28655 raft_consensus.cc:374] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.377692 28655 raft_consensus.cc:729] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.377712 28655 consensus_queue.cc:260] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.377785 28655 ts_tablet_manager.cc:1398] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.377835 28655 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.378175 28655 tablet_bootstrap.cc:654] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.378185 28656 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.378221 28656 ts_tablet_manager.cc:1367] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240503 02:09:18.378348 28656 raft_consensus.cc:348] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.378399 28656 raft_consensus.cc:374] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.378417 28656 raft_consensus.cc:729] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.378456 28656 consensus_queue.cc:260] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.378547 28656 ts_tablet_manager.cc:1398] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.378595 28656 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.378927 28656 tablet_bootstrap.cc:654] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.379904 28656 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.379948 28656 ts_tablet_manager.cc:1367] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.379981 28655 tablet_bootstrap.cc:492] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.380019 28655 ts_tablet_manager.cc:1367] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.380070 28656 raft_consensus.cc:348] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.380116 28656 raft_consensus.cc:374] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.380132 28656 raft_consensus.cc:729] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.380157 28656 consensus_queue.cc:260] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.380141 28655 raft_consensus.cc:348] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.380209 28655 raft_consensus.cc:374] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.380227 28655 raft_consensus.cc:729] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.380241 28656 ts_tablet_manager.cc:1398] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.380259 28655 consensus_queue.cc:260] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.380287 28656 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.380371 28655 ts_tablet_manager.cc:1398] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.380427 28655 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.380615 28656 tablet_bootstrap.cc:654] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.380769 28655 tablet_bootstrap.cc:654] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.381793 29046 mvcc.cc:205] Tried to move back new op lower bound from 7023420040606175232 to 7023420039952060416. Current Snapshot: MvccSnapshot[applied={T|T < 7023420040606175232}]
I20240503 02:09:18.381857 30190 tablet.cc:1722] T b24a6595e7ea4b13a2cb7e5894bdc5d3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.381841 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9dfb0c37a6524daf9700c7ee14eb796f" } modify_external_catalogs: true
I20240503 02:09:18.381901 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9dfb0c37a6524daf9700c7ee14eb796f" } modify_external_catalogs: true
I20240503 02:09:18.382793 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.383462 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3faa2ff595844bd3a1aa8636c70a64b7
I20240503 02:09:18.383688 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.383792 28653 tablet_replica.cc:330] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.383862 28653 raft_consensus.cc:2229] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.383893 28653 raft_consensus.cc:2258] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.384058 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.384089 28653 ts_tablet_manager.cc:1875] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.384145 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3faa2ff595844bd3a1aa8636c70a64b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.384222 28661 tablet_replica.cc:330] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.384285 28661 raft_consensus.cc:2229] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.384311 28655 tablet_bootstrap.cc:492] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.384316 28661 raft_consensus.cc:2258] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.384343 28655 ts_tablet_manager.cc:1367] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:18.384481 28654 tablet_replica.cc:330] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.384541 28654 raft_consensus.cc:2229] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.384536 28655 raft_consensus.cc:348] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.384579 28654 raft_consensus.cc:2258] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.384585 28655 raft_consensus.cc:374] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.384603 28655 raft_consensus.cc:729] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.384639 28655 consensus_queue.cc:260] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.384718 28655 ts_tablet_manager.cc:1398] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.384754 28661 ts_tablet_manager.cc:1875] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.384786 28655 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.385114 28655 tablet_bootstrap.cc:654] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.385563 28653 ts_tablet_manager.cc:1888] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.385609 28653 log.cc:1196] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3faa2ff595844bd3a1aa8636c70a64b7
I20240503 02:09:18.385743 28653 ts_tablet_manager.cc:1909] T 3faa2ff595844bd3a1aa8636c70a64b7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.386148 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_137a9798ebb04facb2c5307cec82768f" } modify_external_catalogs: true
I20240503 02:09:18.386168 28661 ts_tablet_manager.cc:1888] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.386207 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_137a9798ebb04facb2c5307cec82768f" } modify_external_catalogs: true
I20240503 02:09:18.386219 28661 log.cc:1196] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3faa2ff595844bd3a1aa8636c70a64b7
I20240503 02:09:18.386296 28654 ts_tablet_manager.cc:1875] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.386399 28661 ts_tablet_manager.cc:1909] T 3faa2ff595844bd3a1aa8636c70a64b7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.386724 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3faa2ff595844bd3a1aa8636c70a64b7 (table default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]) successfully deleted
I20240503 02:09:18.387331 28656 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.387380 28656 ts_tablet_manager.cc:1367] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.387516 28656 raft_consensus.cc:348] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.387606 28656 raft_consensus.cc:374] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.387627 28656 raft_consensus.cc:729] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.387652 28656 consensus_queue.cc:260] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.387676 28654 ts_tablet_manager.cc:1888] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.387753 28654 log.cc:1196] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3faa2ff595844bd3a1aa8636c70a64b7
I20240503 02:09:18.387758 28656 ts_tablet_manager.cc:1398] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:18.388021 28654 ts_tablet_manager.cc:1909] T 3faa2ff595844bd3a1aa8636c70a64b7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.388345 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.388505 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3faa2ff595844bd3a1aa8636c70a64b7 (table default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]) successfully deleted
I20240503 02:09:18.388770 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet a892df6180d248d0b799c34596f382d9
I20240503 02:09:18.388947 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet a892df6180d248d0b799c34596f382d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.389019 28661 tablet_replica.cc:330] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.389063 28661 raft_consensus.cc:2229] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.389122 28661 raft_consensus.cc:2258] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.389227 28661 ts_tablet_manager.cc:1875] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.389497 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet a892df6180d248d0b799c34596f382d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.389549 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3faa2ff595844bd3a1aa8636c70a64b7 (table default.Table_9dfb0c37a6524daf9700c7ee14eb796f [id=7cb8cab85d0a4a20a93df4ae9263caeb]) successfully deleted
I20240503 02:09:18.389564 27848 consensus_queue.cc:1035] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.389716 28653 tablet_replica.cc:330] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.389772 28653 raft_consensus.cc:2229] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.389807 28653 raft_consensus.cc:2258] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.389897 28653 ts_tablet_manager.cc:1875] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.390421 28661 ts_tablet_manager.cc:1888] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.390460 28661 log.cc:1196] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/a892df6180d248d0b799c34596f382d9
I20240503 02:09:18.390594 28661 ts_tablet_manager.cc:1909] T a892df6180d248d0b799c34596f382d9 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.390686 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet a892df6180d248d0b799c34596f382d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.390786 28654 tablet_replica.cc:330] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.390841 28654 raft_consensus.cc:2229] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.390877 28654 raft_consensus.cc:2258] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.391005 28654 ts_tablet_manager.cc:1875] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.391304 27848 consensus_queue.cc:1035] T 32d2fdc79fec4c60803eb4f9edb792e5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.391324 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.392262 28654 ts_tablet_manager.cc:1888] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.392305 28654 log.cc:1196] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/a892df6180d248d0b799c34596f382d9
I20240503 02:09:18.392495 28654 ts_tablet_manager.cc:1909] T a892df6180d248d0b799c34596f382d9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.392508 28653 ts_tablet_manager.cc:1888] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.392549 28653 log.cc:1196] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/a892df6180d248d0b799c34596f382d9
I20240503 02:09:18.392720 28653 ts_tablet_manager.cc:1909] T a892df6180d248d0b799c34596f382d9 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.392833 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet a892df6180d248d0b799c34596f382d9 (table default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]) successfully deleted
I20240503 02:09:18.393594 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet a892df6180d248d0b799c34596f382d9 (table default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]) successfully deleted
I20240503 02:09:18.394130 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet a892df6180d248d0b799c34596f382d9 (table default.Table_137a9798ebb04facb2c5307cec82768f [id=8695c23a78804a62b2290ed21622a94b]) successfully deleted
I20240503 02:09:18.394534 28655 tablet_bootstrap.cc:492] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.394575 28655 ts_tablet_manager.cc:1367] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240503 02:09:18.394690 28655 raft_consensus.cc:348] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.394735 28655 raft_consensus.cc:374] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.394752 28655 raft_consensus.cc:729] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.394791 28655 consensus_queue.cc:260] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.394872 28655 ts_tablet_manager.cc:1398] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.394943 28655 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.395069 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 8127ac16597e40fa9b1590a61430aea0 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.395411 28655 tablet_bootstrap.cc:654] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.396436 28655 tablet_bootstrap.cc:492] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.396485 28655 ts_tablet_manager.cc:1367] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.396617 28655 raft_consensus.cc:348] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.396675 28655 raft_consensus.cc:374] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.396692 28655 raft_consensus.cc:729] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.396728 28655 consensus_queue.cc:260] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.396816 28655 ts_tablet_manager.cc:1398] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.396876 28655 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.397228 28655 tablet_bootstrap.cc:654] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.397789 28655 tablet_bootstrap.cc:492] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.397835 28655 ts_tablet_manager.cc:1367] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.397950 28655 raft_consensus.cc:348] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.397997 28655 raft_consensus.cc:374] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.398012 28655 raft_consensus.cc:729] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.398044 28655 consensus_queue.cc:260] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.398113 28655 ts_tablet_manager.cc:1398] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.412330 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet dba5e73b037d400ca2417c21f81729a6 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.413722 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet dba5e73b037d400ca2417c21f81729a6
I20240503 02:09:18.413854 27195 catalog_manager.cc:6554] ReplaceTablet: tablet dba5e73b037d400ca2417c21f81729a6 deleted and replaced by tablet 106f9b05066d46ec83ce7e2129840eee
I20240503 02:09:18.413983 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet dba5e73b037d400ca2417c21f81729a6 with delete_type TABLET_DATA_DELETED (replaced by tablet 106f9b05066d46ec83ce7e2129840eee) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.413983 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet dba5e73b037d400ca2417c21f81729a6 with delete_type TABLET_DATA_DELETED (replaced by tablet 106f9b05066d46ec83ce7e2129840eee) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.414074 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet dba5e73b037d400ca2417c21f81729a6 with delete_type TABLET_DATA_DELETED (replaced by tablet 106f9b05066d46ec83ce7e2129840eee) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.414083 28653 tablet_replica.cc:330] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.414155 28654 tablet_replica.cc:330] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.414158 28653 raft_consensus.cc:2229] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.414201 28653 raft_consensus.cc:2258] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.414209 28661 tablet_replica.cc:330] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.414225 28654 raft_consensus.cc:2229] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.414258 28654 raft_consensus.cc:2258] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.414263 28661 raft_consensus.cc:2229] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.414317 28661 raft_consensus.cc:2258] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.414479 28654 ts_tablet_manager.cc:1875] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.414613 28653 ts_tablet_manager.cc:1875] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.415226 28661 ts_tablet_manager.cc:1875] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.415341 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 106f9b05066d46ec83ce7e2129840eee (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.415428 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106f9b05066d46ec83ce7e2129840eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.415894 28654 ts_tablet_manager.cc:1888] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.415937 28654 log.cc:1196] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/dba5e73b037d400ca2417c21f81729a6
I20240503 02:09:18.415956 28653 ts_tablet_manager.cc:1888] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.416009 28653 log.cc:1196] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/dba5e73b037d400ca2417c21f81729a6
I20240503 02:09:18.416123 28654 ts_tablet_manager.cc:1909] T dba5e73b037d400ca2417c21f81729a6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.416198 28653 ts_tablet_manager.cc:1909] T dba5e73b037d400ca2417c21f81729a6 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.416529 28656 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.416625 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet dba5e73b037d400ca2417c21f81729a6 (table default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]) successfully deleted
I20240503 02:09:18.416721 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 106f9b05066d46ec83ce7e2129840eee (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.416800 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106f9b05066d46ec83ce7e2129840eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.417057 28656 tablet_bootstrap.cc:654] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.417562 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 106f9b05066d46ec83ce7e2129840eee (DEFAULT_TABLE table=default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.417649 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 106f9b05066d46ec83ce7e2129840eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.417763 28656 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.417798 28656 ts_tablet_manager.cc:1367] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.417910 28656 raft_consensus.cc:348] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.417963 28656 raft_consensus.cc:374] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.417956 28655 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.417984 28656 raft_consensus.cc:729] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.418030 28656 consensus_queue.cc:260] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.418155 28656 ts_tablet_manager.cc:1398] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.418421 28655 tablet_bootstrap.cc:654] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.418809 28664 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.419164 28664 tablet_bootstrap.cc:654] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.419261 28661 ts_tablet_manager.cc:1888] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.419298 28661 log.cc:1196] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/dba5e73b037d400ca2417c21f81729a6
I20240503 02:09:18.419466 28661 ts_tablet_manager.cc:1909] T dba5e73b037d400ca2417c21f81729a6 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.419706 28664 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.419737 28664 ts_tablet_manager.cc:1367] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.419786 28655 tablet_bootstrap.cc:492] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.419827 28655 ts_tablet_manager.cc:1367] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:18.419884 28664 raft_consensus.cc:348] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.419965 28664 raft_consensus.cc:374] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.419983 28664 raft_consensus.cc:729] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.419963 28655 raft_consensus.cc:348] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.420025 28655 raft_consensus.cc:374] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.420018 28664 consensus_queue.cc:260] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.420046 28655 raft_consensus.cc:729] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.420085 28655 consensus_queue.cc:260] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.420116 28664 ts_tablet_manager.cc:1398] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.420182 28655 ts_tablet_manager.cc:1398] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:18.420318 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet dba5e73b037d400ca2417c21f81729a6 (table default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]) successfully deleted
I20240503 02:09:18.420660 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet dba5e73b037d400ca2417c21f81729a6 (table default.Table_0210f27a4c6348b9bd2efc147d1fc688 [id=393b28da238d467aa73a155141148c2e]) successfully deleted
I20240503 02:09:18.423749 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet fd9df0eb41d54a7ca550c0323c305424 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.423921 27853 raft_consensus.cc:482] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.423966 27853 raft_consensus.cc:504] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.424067 27853 leader_election.cc:288] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.424212 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "029b393711d7465299f648174787dd60" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.424273 27582 raft_consensus.cc:2454] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.424383 27656 leader_election.cc:302] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.424448 27853 raft_consensus.cc:2781] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.424470 27853 raft_consensus.cc:482] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.424486 27853 raft_consensus.cc:3037] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.424772 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet fd9df0eb41d54a7ca550c0323c305424
I20240503 02:09:18.424860 27195 catalog_manager.cc:6554] ReplaceTablet: tablet fd9df0eb41d54a7ca550c0323c305424 deleted and replaced by tablet 17fb873c91f543cf9297c29528eb5ed1
I20240503 02:09:18.425150 27853 raft_consensus.cc:504] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.425244 27853 leader_election.cc:288] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.425287 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet fd9df0eb41d54a7ca550c0323c305424 with delete_type TABLET_DATA_DELETED (replaced by tablet 17fb873c91f543cf9297c29528eb5ed1) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.425416 28653 tablet_replica.cc:330] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.425477 28653 raft_consensus.cc:2229] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.425513 28653 raft_consensus.cc:2258] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.425529 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "029b393711d7465299f648174787dd60" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.425586 27453 raft_consensus.cc:2454] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.425637 28653 ts_tablet_manager.cc:1875] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.425788 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "029b393711d7465299f648174787dd60" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.425843 27582 raft_consensus.cc:3037] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.425884 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet fd9df0eb41d54a7ca550c0323c305424 with delete_type TABLET_DATA_DELETED (replaced by tablet 17fb873c91f543cf9297c29528eb5ed1) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.426533 27582 raft_consensus.cc:2454] T 029b393711d7465299f648174787dd60 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.426661 27656 leader_election.cc:302] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.426822 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 17fb873c91f543cf9297c29528eb5ed1 (DEFAULT_TABLE table=default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.426920 28653 ts_tablet_manager.cc:1888] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.426975 28653 log.cc:1196] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/fd9df0eb41d54a7ca550c0323c305424
I20240503 02:09:18.426934 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fb873c91f543cf9297c29528eb5ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.427016 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 17fb873c91f543cf9297c29528eb5ed1 (DEFAULT_TABLE table=default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.427049 27853 raft_consensus.cc:2781] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.427094 27853 raft_consensus.cc:686] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.427096 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fb873c91f543cf9297c29528eb5ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.427129 27853 consensus_queue.cc:237] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.427179 28653 ts_tablet_manager.cc:1909] T fd9df0eb41d54a7ca550c0323c305424 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.425644 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "029b393711d7465299f648174787dd60" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.427443 27447 raft_consensus.cc:3037] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.428042 27447 raft_consensus.cc:2454] T 029b393711d7465299f648174787dd60 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.428232 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet fd9df0eb41d54a7ca550c0323c305424 with delete_type TABLET_DATA_DELETED (replaced by tablet 17fb873c91f543cf9297c29528eb5ed1) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.428296 28656 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.428473 28655 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.428522 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 17fb873c91f543cf9297c29528eb5ed1 (DEFAULT_TABLE table=default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.428503 27195 catalog_manager.cc:5484] T 029b393711d7465299f648174787dd60 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.428602 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17fb873c91f543cf9297c29528eb5ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.428653 28656 tablet_bootstrap.cc:654] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.428685 28654 tablet_replica.cc:330] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.428735 28654 raft_consensus.cc:2229] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.428763 28654 raft_consensus.cc:2258] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.428790 28655 tablet_bootstrap.cc:654] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.428874 28654 ts_tablet_manager.cc:1875] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.429772 28655 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.429828 28655 ts_tablet_manager.cc:1367] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.429950 28655 raft_consensus.cc:348] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.429996 28655 raft_consensus.cc:374] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.430011 28655 raft_consensus.cc:729] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.430043 28655 consensus_queue.cc:260] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.430140 28655 ts_tablet_manager.cc:1398] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.430145 28654 ts_tablet_manager.cc:1888] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.430186 28654 log.cc:1196] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/fd9df0eb41d54a7ca550c0323c305424
I20240503 02:09:18.430195 28766 consensus_queue.cc:1035] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.430758 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]) successfully deleted
I20240503 02:09:18.431264 28654 ts_tablet_manager.cc:1909] T fd9df0eb41d54a7ca550c0323c305424 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.431377 28664 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.431622 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]) successfully deleted
I20240503 02:09:18.431766 28664 tablet_bootstrap.cc:654] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.432197 28656 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.432248 28656 ts_tablet_manager.cc:1367] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:18.432369 28656 raft_consensus.cc:348] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.432423 28656 raft_consensus.cc:374] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.432440 28656 raft_consensus.cc:729] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.432466 28656 consensus_queue.cc:260] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.432559 28656 ts_tablet_manager.cc:1398] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.432682 28661 tablet_replica.cc:330] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.432763 28661 raft_consensus.cc:2229] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.432829 28661 raft_consensus.cc:2258] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.433032 28661 ts_tablet_manager.cc:1875] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.433130 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
W20240503 02:09:18.433209 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 17fb873c91f543cf9297c29528eb5ed1 consensus state has no leader
I20240503 02:09:18.434576 27195 master_service.cc:946] ReplaceTablet: received request to replace tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.435472 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab
I20240503 02:09:18.435489 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:18.435550 27195 catalog_manager.cc:6554] ReplaceTablet: tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab deleted and replaced by tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7
I20240503 02:09:18.435696 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab with delete_type TABLET_DATA_DELETED (replaced by tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.435715 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab with delete_type TABLET_DATA_DELETED (replaced by tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.435766 28654 tablet_replica.cc:330] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.435808 28654 raft_consensus.cc:2229] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.435855 28654 raft_consensus.cc:2258] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.435953 28653 tablet_replica.cc:330] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.436015 28653 raft_consensus.cc:2229] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.436045 28653 raft_consensus.cc:2258] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.436053 28654 ts_tablet_manager.cc:1875] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.436249 28653 ts_tablet_manager.cc:1875] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.436403 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab with delete_type TABLET_DATA_DELETED (replaced by tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.437228 28654 ts_tablet_manager.cc:1888] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.437263 28654 log.cc:1196] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/333f36f9ab6b4fa28f6fb02a48bfd2ab
I20240503 02:09:18.437449 28653 ts_tablet_manager.cc:1888] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.437484 28654 ts_tablet_manager.cc:1909] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.437496 28653 log.cc:1196] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/333f36f9ab6b4fa28f6fb02a48bfd2ab
I20240503 02:09:18.437675 28653 ts_tablet_manager.cc:1909] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.437678 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet ee57ef1446e246ba98a59945fbcad76e from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.437758 28661 ts_tablet_manager.cc:1888] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.437803 28661 log.cc:1196] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/fd9df0eb41d54a7ca550c0323c305424
I20240503 02:09:18.437911 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:18.437944 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:18.438262 28661 ts_tablet_manager.cc:1909] T fd9df0eb41d54a7ca550c0323c305424 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.438647 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.438743 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.438781 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.438876 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.439003 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7 (DEFAULT_TABLE table=default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.439085 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dcf1601f3ea485ca4ad4a58cf0c7ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.439106 28664 tablet_bootstrap.cc:492] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.439139 28664 ts_tablet_manager.cc:1367] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:18.439260 28664 raft_consensus.cc:348] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.439306 28664 raft_consensus.cc:374] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.439322 28664 raft_consensus.cc:729] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.439356 28664 consensus_queue.cc:260] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.439456 28664 ts_tablet_manager.cc:1398] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.439955 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ee57ef1446e246ba98a59945fbcad76e
I20240503 02:09:18.440038 27198 catalog_manager.cc:6554] ReplaceTablet: tablet ee57ef1446e246ba98a59945fbcad76e deleted and replaced by tablet c270bc02d9b04ddab310b305428fd48e
I20240503 02:09:18.440167 28655 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.440217 28656 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.440289 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet ee57ef1446e246ba98a59945fbcad76e with delete_type TABLET_DATA_DELETED (replaced by tablet c270bc02d9b04ddab310b305428fd48e) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.440524 28655 tablet_bootstrap.cc:654] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.440527 28656 tablet_bootstrap.cc:654] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.440874 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet ee57ef1446e246ba98a59945fbcad76e with delete_type TABLET_DATA_DELETED (replaced by tablet c270bc02d9b04ddab310b305428fd48e) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.440979 28654 tablet_replica.cc:330] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.441030 28654 raft_consensus.cc:2229] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.441059 28654 raft_consensus.cc:2258] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.441155 28654 ts_tablet_manager.cc:1875] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.441193 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet ee57ef1446e246ba98a59945fbcad76e with delete_type TABLET_DATA_DELETED (replaced by tablet c270bc02d9b04ddab310b305428fd48e) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.441479 28653 tablet_replica.cc:330] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.441537 28653 raft_consensus.cc:2229] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.441584 28653 raft_consensus.cc:2258] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.441670 28653 ts_tablet_manager.cc:1875] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.441884 28661 tablet_replica.cc:330] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.441941 28661 raft_consensus.cc:2229] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.441972 28661 raft_consensus.cc:2258] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.442194 28661 ts_tablet_manager.cc:1875] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.442852 28653 ts_tablet_manager.cc:1888] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.442902 28653 log.cc:1196] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ee57ef1446e246ba98a59945fbcad76e
I20240503 02:09:18.443037 28654 ts_tablet_manager.cc:1888] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.443073 28653 ts_tablet_manager.cc:1909] T ee57ef1446e246ba98a59945fbcad76e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.443078 28654 log.cc:1196] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ee57ef1446e246ba98a59945fbcad76e
I20240503 02:09:18.443240 28654 ts_tablet_manager.cc:1909] T ee57ef1446e246ba98a59945fbcad76e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.443231 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet fd9df0eb41d54a7ca550c0323c305424 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]) successfully deleted
I20240503 02:09:18.443467 28661 ts_tablet_manager.cc:1888] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.443543 28661 log.cc:1196] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/333f36f9ab6b4fa28f6fb02a48bfd2ab
I20240503 02:09:18.443542 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ee57ef1446e246ba98a59945fbcad76e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.443625 28664 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.443738 28655 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.443780 28655 ts_tablet_manager.cc:1367] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:18.443862 28661 ts_tablet_manager.cc:1909] T 333f36f9ab6b4fa28f6fb02a48bfd2ab P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.443907 28655 raft_consensus.cc:348] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.443961 28655 raft_consensus.cc:374] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.443977 28655 raft_consensus.cc:729] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.443984 28664 tablet_bootstrap.cc:654] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.444031 28655 consensus_queue.cc:260] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.444124 28655 ts_tablet_manager.cc:1398] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.444152 28661 tablet_replica.cc:330] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.444203 28661 raft_consensus.cc:2229] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.444213 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 333f36f9ab6b4fa28f6fb02a48bfd2ab (table default.Table_03f97ede69bd47fdb581e34a506b8dea [id=6ba9fa58ba1d4884a896ea347d557de3]) successfully deleted
I20240503 02:09:18.444231 28661 raft_consensus.cc:2258] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.444332 28661 ts_tablet_manager.cc:1875] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.444355 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ee57ef1446e246ba98a59945fbcad76e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.444806 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:18.445158 27422 tablet_service.cc:1467] Processing CreateTablet for tablet c270bc02d9b04ddab310b305428fd48e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.445186 27554 tablet_service.cc:1467] Processing CreateTablet for tablet c270bc02d9b04ddab310b305428fd48e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.445262 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c270bc02d9b04ddab310b305428fd48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.445274 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c270bc02d9b04ddab310b305428fd48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.445497 28656 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.445538 28656 ts_tablet_manager.cc:1367] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.445654 28656 raft_consensus.cc:348] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.445709 28656 raft_consensus.cc:374] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.445726 28656 raft_consensus.cc:729] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.445775 28656 consensus_queue.cc:260] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.445875 28656 ts_tablet_manager.cc:1398] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.446118 28664 tablet_bootstrap.cc:492] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.446128 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_97f6a0dc82a840dd82a2fc7b3591801d" } modify_external_catalogs: true
I20240503 02:09:18.446162 28664 ts_tablet_manager.cc:1367] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240503 02:09:18.446187 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_97f6a0dc82a840dd82a2fc7b3591801d" } modify_external_catalogs: true
I20240503 02:09:18.446293 28664 raft_consensus.cc:348] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.446345 28664 raft_consensus.cc:374] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.446362 28664 raft_consensus.cc:729] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.446396 28664 consensus_queue.cc:260] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.446489 28664 ts_tablet_manager.cc:1398] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.447614 28766 raft_consensus.cc:482] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.447664 28766 raft_consensus.cc:504] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.447763 28766 leader_election.cc:288] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.447911 28656 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.447907 28655 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.448078 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd59c2a068642c6af4742c990176172" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.448135 27722 raft_consensus.cc:2454] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.448271 28655 tablet_bootstrap.cc:654] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.448267 28656 tablet_bootstrap.cc:654] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.448694 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd59c2a068642c6af4742c990176172" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.448762 27586 raft_consensus.cc:2454] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.448801 27388 leader_election.cc:302] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.448873 27703 tablet_service.cc:1467] Processing CreateTablet for tablet c270bc02d9b04ddab310b305428fd48e (DEFAULT_TABLE table=default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.448959 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c270bc02d9b04ddab310b305428fd48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.449143 28767 raft_consensus.cc:2781] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.449179 28767 raft_consensus.cc:482] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.449196 28767 raft_consensus.cc:3037] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.449820 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 0d4a1909677544549029137b79501235
I20240503 02:09:18.449844 28767 raft_consensus.cc:504] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.449934 28767 leader_election.cc:288] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.450126 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d4a1909677544549029137b79501235 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.450191 28654 tablet_replica.cc:330] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.450245 28654 raft_consensus.cc:2229] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.450297 28654 raft_consensus.cc:2258] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.450368 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d4a1909677544549029137b79501235 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.450402 28654 ts_tablet_manager.cc:1875] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.450490 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd59c2a068642c6af4742c990176172" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.450559 27584 raft_consensus.cc:3037] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.450953 28661 ts_tablet_manager.cc:1888] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.450995 28661 log.cc:1196] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ee57ef1446e246ba98a59945fbcad76e
I20240503 02:09:18.451171 28661 ts_tablet_manager.cc:1909] T ee57ef1446e246ba98a59945fbcad76e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.451169 27584 raft_consensus.cc:2454] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.451649 27386 leader_election.cc:302] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.451668 28654 ts_tablet_manager.cc:1888] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.451704 28654 log.cc:1196] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0d4a1909677544549029137b79501235
I20240503 02:09:18.451736 28766 raft_consensus.cc:2781] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.451771 28766 raft_consensus.cc:686] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.451783 28661 tablet_replica.cc:330] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.451813 28766 consensus_queue.cc:237] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.451900 28654 ts_tablet_manager.cc:1909] T 0d4a1909677544549029137b79501235 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.451931 28661 raft_consensus.cc:2229] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.451974 28661 raft_consensus.cc:2258] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.452015 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd59c2a068642c6af4742c990176172" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.452075 27722 raft_consensus.cc:3037] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.452214 28661 ts_tablet_manager.cc:1875] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.452303 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0d4a1909677544549029137b79501235 (table default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81]) successfully deleted
I20240503 02:09:18.452397 28664 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.452440 27718 raft_consensus.cc:3037] T 95d96daa3de7403a865d872e8d678628 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.452572 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9d7102980a8a4f368a1d2997b60e0fc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.452692 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d7102980a8a4f368a1d2997b60e0fc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.452688 27722 raft_consensus.cc:2454] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.452755 28664 tablet_bootstrap.cc:654] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.453156 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ee57ef1446e246ba98a59945fbcad76e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.453171 29550 consensus_queue.cc:1035] T 95d96daa3de7403a865d872e8d678628 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.453621 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d4a1909677544549029137b79501235 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.453859 28653 tablet_replica.cc:330] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.453935 28653 raft_consensus.cc:2229] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.453991 28653 raft_consensus.cc:2258] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.454089 28653 ts_tablet_manager.cc:1875] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.454494 28661 ts_tablet_manager.cc:1888] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.454545 28661 log.cc:1196] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0d4a1909677544549029137b79501235
I20240503 02:09:18.454809 28661 ts_tablet_manager.cc:1909] T 0d4a1909677544549029137b79501235 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.455087 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0d4a1909677544549029137b79501235 (table default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81]) successfully deleted
I20240503 02:09:18.455292 28655 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.455334 28655 ts_tablet_manager.cc:1367] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.455366 28653 ts_tablet_manager.cc:1888] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.455415 28653 log.cc:1196] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0d4a1909677544549029137b79501235
I20240503 02:09:18.455468 28655 raft_consensus.cc:348] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.455515 28655 raft_consensus.cc:374] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.455530 28655 raft_consensus.cc:729] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.455565 28655 consensus_queue.cc:260] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.455593 28653 ts_tablet_manager.cc:1909] T 0d4a1909677544549029137b79501235 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.455652 28655 ts_tablet_manager.cc:1398] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.457494 27195 catalog_manager.cc:5484] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.458829 28410 consensus_queue.cc:1035] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.458967 28656 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.459026 28656 ts_tablet_manager.cc:1367] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.000s
I20240503 02:09:18.459225 28656 raft_consensus.cc:348] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.459287 28656 raft_consensus.cc:374] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.459307 28656 raft_consensus.cc:729] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.459345 28656 consensus_queue.cc:260] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.459437 28656 ts_tablet_manager.cc:1398] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.461120 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 16e0cc2ff5594a28af52d056c6d6942d (DEFAULT_TABLE table=default.Table_9d7102980a8a4f368a1d2997b60e0fc0 [id=ae807eba5c7447ad837d75dd12ed74e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.461222 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e0cc2ff5594a28af52d056c6d6942d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.461364 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 16e0cc2ff5594a28af52d056c6d6942d (DEFAULT_TABLE table=default.Table_9d7102980a8a4f368a1d2997b60e0fc0 [id=ae807eba5c7447ad837d75dd12ed74e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.461465 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e0cc2ff5594a28af52d056c6d6942d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.462536 28410 consensus_queue.cc:1035] T 2fd1cb352c40442696afd2fc0bcac0fd P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.462611 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 16e0cc2ff5594a28af52d056c6d6942d (DEFAULT_TABLE table=default.Table_9d7102980a8a4f368a1d2997b60e0fc0 [id=ae807eba5c7447ad837d75dd12ed74e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.462697 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16e0cc2ff5594a28af52d056c6d6942d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.463745 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0d4a1909677544549029137b79501235 (table default.Table_97f6a0dc82a840dd82a2fc7b3591801d [id=3b9e6ecebf8b4abe9b44c085ac626a81]) successfully deleted
I20240503 02:09:18.463801 28655 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.464159 28655 tablet_bootstrap.cc:654] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.464361 28656 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.464722 28656 tablet_bootstrap.cc:654] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.465234 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4ff567fc40464d588787c5b5390948ac" } modify_external_catalogs: true
I20240503 02:09:18.465301 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_4ff567fc40464d588787c5b5390948ac" } modify_external_catalogs: true
I20240503 02:09:18.465395 28655 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.465435 28655 ts_tablet_manager.cc:1367] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.465561 28655 raft_consensus.cc:348] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.465615 28655 raft_consensus.cc:374] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.465632 28655 raft_consensus.cc:729] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.465655 28655 consensus_queue.cc:260] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.465737 28655 ts_tablet_manager.cc:1398] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.466462 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 3ea94da2559e423a9dbb0cb8bc50cb8b
I20240503 02:09:18.466689 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.466917 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.467363 28654 tablet_replica.cc:330] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.467437 28654 raft_consensus.cc:2229] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.467497 28654 raft_consensus.cc:2258] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.467700 28656 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.467737 28656 ts_tablet_manager.cc:1367] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:18.467860 28656 raft_consensus.cc:348] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.467911 28656 raft_consensus.cc:374] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.467929 28656 raft_consensus.cc:729] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.467972 28656 consensus_queue.cc:260] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.468053 28656 ts_tablet_manager.cc:1398] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.468092 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea94da2559e423a9dbb0cb8bc50cb8b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.468161 28653 tablet_replica.cc:330] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.468212 28653 raft_consensus.cc:2229] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.468238 28653 raft_consensus.cc:2258] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.468390 28654 ts_tablet_manager.cc:1875] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.468492 28653 ts_tablet_manager.cc:1875] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.468866 28664 tablet_bootstrap.cc:492] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.468909 28664 ts_tablet_manager.cc:1367] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240503 02:09:18.469033 28664 raft_consensus.cc:348] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.469071 28664 raft_consensus.cc:374] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.469082 28664 raft_consensus.cc:729] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.469116 28664 consensus_queue.cc:260] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.469203 28664 ts_tablet_manager.cc:1398] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:18.469287 27524 consensus_peers.cc:485] T 85e1f031e13f47c692de9a8a5e256613 P 5cc801b4e0b047b3b4b30f8dd5c7c5db -> Peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Couldn't send request to peer bd4b79bd8b57443faa6fa5e464570188. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85e1f031e13f47c692de9a8a5e256613. This is attempt 11: this message will repeat every 5th retry.
I20240503 02:09:18.469400 28664 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.469764 28664 tablet_bootstrap.cc:654] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.469791 28654 ts_tablet_manager.cc:1888] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.469833 28654 log.cc:1196] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/3ea94da2559e423a9dbb0cb8bc50cb8b
I20240503 02:09:18.470021 28654 ts_tablet_manager.cc:1909] T 3ea94da2559e423a9dbb0cb8bc50cb8b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.470615 28653 ts_tablet_manager.cc:1888] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.470662 28653 log.cc:1196] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/3ea94da2559e423a9dbb0cb8bc50cb8b
I20240503 02:09:18.470832 28664 tablet_bootstrap.cc:492] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.470844 28653 ts_tablet_manager.cc:1909] T 3ea94da2559e423a9dbb0cb8bc50cb8b P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.470870 28664 ts_tablet_manager.cc:1367] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.470996 28664 raft_consensus.cc:348] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.471063 28664 raft_consensus.cc:374] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.471081 28664 raft_consensus.cc:729] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.471103 28664 consensus_queue.cc:260] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.471186 28664 ts_tablet_manager.cc:1398] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.471436 28661 tablet_replica.cc:330] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.471510 28661 raft_consensus.cc:2229] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.471544 28661 raft_consensus.cc:2258] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.471786 28661 ts_tablet_manager.cc:1875] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.471854 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (table default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]) successfully deleted
W20240503 02:09:18.472144 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:18.472669 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (table default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]) successfully deleted
I20240503 02:09:18.473104 28661 ts_tablet_manager.cc:1888] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.473145 28661 log.cc:1196] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/3ea94da2559e423a9dbb0cb8bc50cb8b
I20240503 02:09:18.473343 28661 ts_tablet_manager.cc:1909] T 3ea94da2559e423a9dbb0cb8bc50cb8b P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.473662 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 3ea94da2559e423a9dbb0cb8bc50cb8b (table default.Table_4ff567fc40464d588787c5b5390948ac [id=02f770bcfec54d4f896f650e38c8227b]) successfully deleted
I20240503 02:09:18.475165 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_96419dbf8ed4471b9c27a81e9636ed48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.475267 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96419dbf8ed4471b9c27a81e9636ed48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.477454 27703 tablet_service.cc:1467] Processing CreateTablet for tablet e04682e3c79e4f8ea8f045a83f785da6 (DEFAULT_TABLE table=default.Table_96419dbf8ed4471b9c27a81e9636ed48 [id=6fe82938f0fd4bd282495daa05a98f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.477468 27554 tablet_service.cc:1467] Processing CreateTablet for tablet e04682e3c79e4f8ea8f045a83f785da6 (DEFAULT_TABLE table=default.Table_96419dbf8ed4471b9c27a81e9636ed48 [id=6fe82938f0fd4bd282495daa05a98f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.477555 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04682e3c79e4f8ea8f045a83f785da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.477555 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04682e3c79e4f8ea8f045a83f785da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.478682 28656 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.478978 27422 tablet_service.cc:1467] Processing CreateTablet for tablet e04682e3c79e4f8ea8f045a83f785da6 (DEFAULT_TABLE table=default.Table_96419dbf8ed4471b9c27a81e9636ed48 [id=6fe82938f0fd4bd282495daa05a98f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.479069 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e04682e3c79e4f8ea8f045a83f785da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.479068 28656 tablet_bootstrap.cc:654] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.479339 28664 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.479514 28656 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.479547 28656 ts_tablet_manager.cc:1367] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.479645 28656 raft_consensus.cc:348] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.479681 28656 raft_consensus.cc:374] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.479692 28656 raft_consensus.cc:729] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.479725 28656 consensus_queue.cc:260] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.479804 28664 tablet_bootstrap.cc:654] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.479811 28656 ts_tablet_manager.cc:1398] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.480216 28655 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.480583 28655 tablet_bootstrap.cc:654] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.481344 28655 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.481393 28655 ts_tablet_manager.cc:1367] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.481518 28655 raft_consensus.cc:348] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.481557 28655 raft_consensus.cc:374] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.481568 28655 raft_consensus.cc:729] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.481601 28655 consensus_queue.cc:260] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.481693 28655 ts_tablet_manager.cc:1398] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.481683 27848 raft_consensus.cc:482] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.481722 27848 raft_consensus.cc:504] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.481825 27848 leader_election.cc:288] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.481922 28664 tablet_bootstrap.cc:492] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.481968 28664 ts_tablet_manager.cc:1367] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20240503 02:09:18.482110 28664 raft_consensus.cc:348] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.482160 28664 raft_consensus.cc:374] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.482177 28664 raft_consensus.cc:729] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.482211 28664 consensus_queue.cc:260] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.482300 28664 ts_tablet_manager.cc:1398] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.482457 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6062b6fee7974681b5fddbea0eb1ecfc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.482529 27447 raft_consensus.cc:2454] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.483171 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6062b6fee7974681b5fddbea0eb1ecfc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.483246 27722 raft_consensus.cc:2454] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.483253 27524 leader_election.cc:302] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.483316 27848 raft_consensus.cc:2781] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.483355 27848 raft_consensus.cc:482] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.483371 27848 raft_consensus.cc:3037] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.483985 27848 raft_consensus.cc:504] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.484079 27848 leader_election.cc:288] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.484233 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6062b6fee7974681b5fddbea0eb1ecfc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.484254 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6062b6fee7974681b5fddbea0eb1ecfc" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.484290 27447 raft_consensus.cc:3037] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.484299 27722 raft_consensus.cc:3037] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.484434 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_eae6a75c43444bd0bdfc97a19e18f149"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.484524 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eae6a75c43444bd0bdfc97a19e18f149 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.484900 27722 raft_consensus.cc:2454] T 6062b6fee7974681b5fddbea0eb1ecfc P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.484900 27447 raft_consensus.cc:2454] T 6062b6fee7974681b5fddbea0eb1ecfc P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.485126 27523 leader_election.cc:302] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.485213 27848 raft_consensus.cc:2781] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.485252 27848 raft_consensus.cc:686] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.485287 27848 consensus_queue.cc:237] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.487557 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 3e040f4320b542cf9c6346447e1a66a4 (DEFAULT_TABLE table=default.Table_eae6a75c43444bd0bdfc97a19e18f149 [id=5823563ba7064bd9a549eb62db229bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.487650 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e040f4320b542cf9c6346447e1a66a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.487726 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 3e040f4320b542cf9c6346447e1a66a4 (DEFAULT_TABLE table=default.Table_eae6a75c43444bd0bdfc97a19e18f149 [id=5823563ba7064bd9a549eb62db229bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.487810 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e040f4320b542cf9c6346447e1a66a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.487836 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 3e040f4320b542cf9c6346447e1a66a4 (DEFAULT_TABLE table=default.Table_eae6a75c43444bd0bdfc97a19e18f149 [id=5823563ba7064bd9a549eb62db229bef]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.487938 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e040f4320b542cf9c6346447e1a66a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.488690 28656 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.488771 28655 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.488783 27197 catalog_manager.cc:5484] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.489086 28656 tablet_bootstrap.cc:654] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.489145 28655 tablet_bootstrap.cc:654] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.489387 28664 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.489773 28664 tablet_bootstrap.cc:654] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.490234 28664 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.490273 28664 ts_tablet_manager.cc:1367] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.490402 28664 raft_consensus.cc:348] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.490463 28664 raft_consensus.cc:374] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.490480 28664 raft_consensus.cc:729] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.490518 28664 consensus_queue.cc:260] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.490500 28507 consensus_queue.cc:1035] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.490603 28664 ts_tablet_manager.cc:1398] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.492445 28656 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.492496 28656 ts_tablet_manager.cc:1367] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240503 02:09:18.492656 28656 raft_consensus.cc:348] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.492719 28656 raft_consensus.cc:374] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.492739 28656 raft_consensus.cc:729] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.492776 28656 consensus_queue.cc:260] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.492794 29643 raft_consensus.cc:482] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.492839 29643 raft_consensus.cc:504] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.492869 28656 ts_tablet_manager.cc:1398] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.492889 28655 tablet_bootstrap.cc:492] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.492928 28655 ts_tablet_manager.cc:1367] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:18.492969 29643 leader_election.cc:288] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.493058 28655 raft_consensus.cc:348] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.493075 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941c3021996a4962b87b068018948e67" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.493104 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941c3021996a4962b87b068018948e67" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.493144 27453 raft_consensus.cc:2454] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.493153 27584 raft_consensus.cc:2454] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.493249 27659 leader_election.cc:302] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.493109 28655 raft_consensus.cc:374] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.493309 28655 raft_consensus.cc:729] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.493371 28655 consensus_queue.cc:260] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.493487 28655 ts_tablet_manager.cc:1398] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.493640 29643 raft_consensus.cc:2781] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.493680 29643 raft_consensus.cc:482] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.493705 29643 raft_consensus.cc:3037] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.494333 29643 raft_consensus.cc:504] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.494439 29643 leader_election.cc:288] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.494649 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941c3021996a4962b87b068018948e67" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.494704 27447 raft_consensus.cc:3037] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.495043 28767 raft_consensus.cc:482] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.495088 28767 raft_consensus.cc:504] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.495132 29240 consensus_queue.cc:1035] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.495179 28767 leader_election.cc:288] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.495297 27447 raft_consensus.cc:2454] T 941c3021996a4962b87b068018948e67 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.495311 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e6a23f6efb49b098699bc6a91e1709" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.495366 27722 raft_consensus.cc:2454] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.495486 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e6a23f6efb49b098699bc6a91e1709" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.495543 27582 raft_consensus.cc:2454] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.495581 27388 leader_election.cc:302] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.495610 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "941c3021996a4962b87b068018948e67" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.495646 27584 raft_consensus.cc:3037] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.496133 27584 raft_consensus.cc:2454] T 941c3021996a4962b87b068018948e67 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.496198 28767 raft_consensus.cc:2781] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.496235 28767 raft_consensus.cc:482] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.496251 28767 raft_consensus.cc:3037] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.496294 27656 leader_election.cc:302] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.496364 28774 raft_consensus.cc:482] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.496369 27853 raft_consensus.cc:2781] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.496409 28774 raft_consensus.cc:504] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.496459 27853 raft_consensus.cc:686] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.496532 28774 leader_election.cc:288] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.496510 27853 consensus_queue.cc:237] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.496639 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c270bc02d9b04ddab310b305428fd48e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.496702 27453 raft_consensus.cc:2454] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.496802 27659 leader_election.cc:302] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.496852 28767 raft_consensus.cc:504] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.496959 28767 leader_election.cc:288] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.496995 27853 raft_consensus.cc:2781] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.497027 27853 raft_consensus.cc:482] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.497045 27853 raft_consensus.cc:3037] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.497618 27853 raft_consensus.cc:504] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.497743 27853 leader_election.cc:288] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.497599 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e6a23f6efb49b098699bc6a91e1709" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.497834 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c270bc02d9b04ddab310b305428fd48e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.497857 27722 raft_consensus.cc:3037] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.497881 27453 raft_consensus.cc:3037] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.498450 27722 raft_consensus.cc:2454] T 40e6a23f6efb49b098699bc6a91e1709 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.498540 27453 raft_consensus.cc:2454] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.498580 27388 leader_election.cc:302] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.498602 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40e6a23f6efb49b098699bc6a91e1709" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.498651 27582 raft_consensus.cc:3037] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.498688 27659 leader_election.cc:302] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.498760 28767 raft_consensus.cc:2781] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.498764 27853 raft_consensus.cc:2781] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.498802 27853 raft_consensus.cc:686] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.498804 28767 raft_consensus.cc:686] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.498840 28767 consensus_queue.cc:237] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.498840 27853 consensus_queue.cc:237] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.499272 27582 raft_consensus.cc:2454] T 40e6a23f6efb49b098699bc6a91e1709 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.499434 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c270bc02d9b04ddab310b305428fd48e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.499485 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c270bc02d9b04ddab310b305428fd48e" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.499588 29240 consensus_queue.cc:1035] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.499491 27582 raft_consensus.cc:3037] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.500357 27582 raft_consensus.cc:2454] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.500970 27198 catalog_manager.cc:5484] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.501084 27198 catalog_manager.cc:5484] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.501144 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_2303e73610bd4cbca13afa55764f55a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.501260 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2303e73610bd4cbca13afa55764f55a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.501425 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_8d143ef35bbc4228bbc4d64bda6bb490"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.501499 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d143ef35bbc4228bbc4d64bda6bb490 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.500401 27197 catalog_manager.cc:5484] T 40e6a23f6efb49b098699bc6a91e1709 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.502728 28410 consensus_queue.cc:1035] T 647ba8fd2b324e8884e22e16ca414471 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.503733 27853 raft_consensus.cc:482] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.503772 27853 raft_consensus.cc:504] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.503861 27853 leader_election.cc:288] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.504048 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106f9b05066d46ec83ce7e2129840eee" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.504109 27584 raft_consensus.cc:2454] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.504285 27656 leader_election.cc:302] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.504360 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106f9b05066d46ec83ce7e2129840eee" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.504397 27853 raft_consensus.cc:2781] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.504423 27853 raft_consensus.cc:482] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.504424 27453 raft_consensus.cc:2454] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.504441 27853 raft_consensus.cc:3037] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.505052 27853 raft_consensus.cc:504] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.505165 27853 leader_election.cc:288] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.505424 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106f9b05066d46ec83ce7e2129840eee" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.505481 27447 raft_consensus.cc:3037] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.506095 27447 raft_consensus.cc:2454] T 106f9b05066d46ec83ce7e2129840eee P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.506306 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "106f9b05066d46ec83ce7e2129840eee" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.506358 27582 raft_consensus.cc:3037] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.506314 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_65120fff74d2408082729cad99366baf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.506422 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65120fff74d2408082729cad99366baf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.506801 27659 leader_election.cc:302] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.506898 27853 raft_consensus.cc:2781] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.506947 27853 raft_consensus.cc:686] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.506973 27582 raft_consensus.cc:2454] T 106f9b05066d46ec83ce7e2129840eee P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.506982 27853 consensus_queue.cc:237] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.508191 27200 catalog_manager.cc:5484] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.508460 27422 tablet_service.cc:1467] Processing CreateTablet for tablet b88e8d58093e42dda61d4a08329c5ee9 (DEFAULT_TABLE table=default.Table_2303e73610bd4cbca13afa55764f55a9 [id=6701a39c2a774b399bcb66fa46189a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.508492 27554 tablet_service.cc:1467] Processing CreateTablet for tablet b88e8d58093e42dda61d4a08329c5ee9 (DEFAULT_TABLE table=default.Table_2303e73610bd4cbca13afa55764f55a9 [id=6701a39c2a774b399bcb66fa46189a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.508562 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88e8d58093e42dda61d4a08329c5ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.508608 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88e8d58093e42dda61d4a08329c5ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.509851 28655 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.509837 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8f5a6141ccf84a0eb4f0bd639d6b3c79" } new_table_name: "default.Table_989c040b372244d2875f77eed4f845ac" modify_external_catalogs: true
I20240503 02:09:18.509949 28656 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.510322 28656 tablet_bootstrap.cc:654] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.510325 28655 tablet_bootstrap.cc:654] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.510885 28655 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.510915 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b88e8d58093e42dda61d4a08329c5ee9 (DEFAULT_TABLE table=default.Table_2303e73610bd4cbca13afa55764f55a9 [id=6701a39c2a774b399bcb66fa46189a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.510938 28655 ts_tablet_manager.cc:1367] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.511024 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88e8d58093e42dda61d4a08329c5ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.511061 28655 raft_consensus.cc:348] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.511112 28655 raft_consensus.cc:374] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.511129 28655 raft_consensus.cc:729] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.511157 28656 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.511165 28655 consensus_queue.cc:260] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.511200 28656 ts_tablet_manager.cc:1367] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.511245 28655 ts_tablet_manager.cc:1398] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.511322 28656 raft_consensus.cc:348] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.511385 28656 raft_consensus.cc:374] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.511404 28656 raft_consensus.cc:729] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.511458 28656 consensus_queue.cc:260] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.511551 28656 ts_tablet_manager.cc:1398] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.512270 28664 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.512670 28664 tablet_bootstrap.cc:654] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.514320 28664 tablet_bootstrap.cc:492] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.514365 28664 ts_tablet_manager.cc:1367] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.514493 28664 raft_consensus.cc:348] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.514544 28664 raft_consensus.cc:374] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.514562 28664 raft_consensus.cc:729] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.514596 28664 consensus_queue.cc:260] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.514678 28664 ts_tablet_manager.cc:1398] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.516038 27422 tablet_service.cc:1467] Processing CreateTablet for tablet a6e343d86baf43ee9264f136015eb7db (DEFAULT_TABLE table=default.Table_8d143ef35bbc4228bbc4d64bda6bb490 [id=781e1e3974e047feb48875450f32ee03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.516141 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e343d86baf43ee9264f136015eb7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.516306 27554 tablet_service.cc:1467] Processing CreateTablet for tablet a6e343d86baf43ee9264f136015eb7db (DEFAULT_TABLE table=default.Table_8d143ef35bbc4228bbc4d64bda6bb490 [id=781e1e3974e047feb48875450f32ee03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.516409 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e343d86baf43ee9264f136015eb7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.516700 30365 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.516855 29643 raft_consensus.cc:482] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.516906 29643 raft_consensus.cc:504] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.517001 29643 leader_election.cc:288] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.517195 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b820e0b556240b393ee31f0eae3a36b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.517266 27447 raft_consensus.cc:2454] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.517345 28655 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.517388 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a6e343d86baf43ee9264f136015eb7db (DEFAULT_TABLE table=default.Table_8d143ef35bbc4228bbc4d64bda6bb490 [id=781e1e3974e047feb48875450f32ee03]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.517480 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e343d86baf43ee9264f136015eb7db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.517518 27659 leader_election.cc:302] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.517596 29643 raft_consensus.cc:2781] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.517621 29643 raft_consensus.cc:482] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.517642 29643 raft_consensus.cc:3037] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.517714 28655 tablet_bootstrap.cc:654] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.518160 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_31b4254047994e7baa9a8657646df1cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.518275 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_31b4254047994e7baa9a8657646df1cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.518365 29643 raft_consensus.cc:504] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.518464 29643 leader_election.cc:288] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.518546 28656 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.518579 28664 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.518729 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b820e0b556240b393ee31f0eae3a36b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.518800 27587 raft_consensus.cc:2454] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.518950 28656 tablet_bootstrap.cc:654] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.519019 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b820e0b556240b393ee31f0eae3a36b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.519065 28664 tablet_bootstrap.cc:654] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.519069 27587 raft_consensus.cc:3037] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.519130 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b820e0b556240b393ee31f0eae3a36b" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.519188 27447 raft_consensus.cc:3037] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.519336 30190 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.519766 27587 raft_consensus.cc:2454] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.519811 27447 raft_consensus.cc:2454] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.519846 28656 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.519881 28656 ts_tablet_manager.cc:1367] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.519935 27656 leader_election.cc:302] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.520006 28656 raft_consensus.cc:348] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.520051 28656 raft_consensus.cc:374] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.520054 27853 raft_consensus.cc:2781] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.520066 28656 raft_consensus.cc:729] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.520092 27853 raft_consensus.cc:686] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.520104 28656 consensus_queue.cc:260] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.520131 27853 consensus_queue.cc:237] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.520208 28656 ts_tablet_manager.cc:1398] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.520207 30370 tablet.cc:1722] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20240503 02:09:18.521303 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 8c502af17069460aa2ddfee4a9ffaed4 (DEFAULT_TABLE table=default.Table_65120fff74d2408082729cad99366baf [id=9d924b81ce3a486a8e8b0091f66959eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.521430 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c502af17069460aa2ddfee4a9ffaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.521960 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_51f3d037d2e4460fa3a131fc639d7eb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.522080 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51f3d037d2e4460fa3a131fc639d7eb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.522830 28656 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.522995 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 3dac6b3737de4d9e8b5063d82b6a4648 (DEFAULT_TABLE table=default.Table_31b4254047994e7baa9a8657646df1cc [id=3d502ea9d602466494e011e694979ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.523018 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 8c502af17069460aa2ddfee4a9ffaed4 (DEFAULT_TABLE table=default.Table_65120fff74d2408082729cad99366baf [id=9d924b81ce3a486a8e8b0091f66959eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.523098 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dac6b3737de4d9e8b5063d82b6a4648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.523098 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c502af17069460aa2ddfee4a9ffaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.523202 28656 tablet_bootstrap.cc:654] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.524255 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 3dac6b3737de4d9e8b5063d82b6a4648 (DEFAULT_TABLE table=default.Table_31b4254047994e7baa9a8657646df1cc [id=3d502ea9d602466494e011e694979ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.524375 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dac6b3737de4d9e8b5063d82b6a4648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.524449 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 8c502af17069460aa2ddfee4a9ffaed4 (DEFAULT_TABLE table=default.Table_65120fff74d2408082729cad99366baf [id=9d924b81ce3a486a8e8b0091f66959eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.524530 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c502af17069460aa2ddfee4a9ffaed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.524683 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_d7d1d67ae02d4f349e3c19d354ad1f74"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.524786 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7d1d67ae02d4f349e3c19d354ad1f74 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.526253 28664 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.526297 28664 ts_tablet_manager.cc:1367] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:18.526305 27198 catalog_manager.cc:5484] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.526404 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.526425 28664 raft_consensus.cc:348] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.526479 28664 raft_consensus.cc:374] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.526494 28664 raft_consensus.cc:729] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.526517 28664 consensus_queue.cc:260] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.526609 28664 ts_tablet_manager.cc:1398] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.526705 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 3dac6b3737de4d9e8b5063d82b6a4648 (DEFAULT_TABLE table=default.Table_31b4254047994e7baa9a8657646df1cc [id=3d502ea9d602466494e011e694979ba7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.526818 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dac6b3737de4d9e8b5063d82b6a4648. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.526965 28655 tablet_bootstrap.cc:492] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.527009 28655 ts_tablet_manager.cc:1367] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:18.527150 28655 raft_consensus.cc:348] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.527207 28655 raft_consensus.cc:374] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.527225 28655 raft_consensus.cc:729] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.527251 28655 consensus_queue.cc:260] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.527348 28655 ts_tablet_manager.cc:1398] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.527451 28655 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.527815 28655 tablet_bootstrap.cc:654] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.528247 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_989c040b372244d2875f77eed4f845ac [id=15574575df68415693e80eaea9955e92] alter complete (version 2)
I20240503 02:09:18.528743 28655 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.528787 28655 ts_tablet_manager.cc:1367] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.528942 28655 raft_consensus.cc:348] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.528995 28655 raft_consensus.cc:374] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.529008 28655 raft_consensus.cc:729] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.529039 28655 consensus_queue.cc:260] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.529117 28655 ts_tablet_manager.cc:1398] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.529166 28655 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.529421 28664 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.529594 28655 tablet_bootstrap.cc:654] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.529814 28664 tablet_bootstrap.cc:654] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.530030 28655 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.530067 28655 ts_tablet_manager.cc:1367] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.530201 28655 raft_consensus.cc:348] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.530251 28655 raft_consensus.cc:374] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.530267 28655 raft_consensus.cc:729] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.530292 28655 consensus_queue.cc:260] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.530386 28655 ts_tablet_manager.cc:1398] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.530974 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ec955a8689124c99a40190db45718286" } modify_external_catalogs: true
I20240503 02:09:18.531059 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ec955a8689124c99a40190db45718286" } modify_external_catalogs: true
I20240503 02:09:18.531633 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a682a7477de54349b2021f1d660a7d1f" } new_table_name: "default.Table_bdb8cc2c2da4465981bf233f470eff28" modify_external_catalogs: true
I20240503 02:09:18.532490 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 50566b0702db4f85b581931f415ca42d
I20240503 02:09:18.532734 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 50566b0702db4f85b581931f415ca42d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.532810 27422 tablet_service.cc:1514] Processing DeleteTablet for tablet 50566b0702db4f85b581931f415ca42d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.532845 28654 tablet_replica.cc:330] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.532860 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 50566b0702db4f85b581931f415ca42d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.532889 28653 tablet_replica.cc:330] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.532930 28661 tablet_replica.cc:330] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.532929 28654 raft_consensus.cc:2229] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.532943 28653 raft_consensus.cc:2229] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.532974 28653 raft_consensus.cc:2258] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.532985 28661 raft_consensus.cc:2229] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.533000 28654 raft_consensus.cc:2258] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.533026 28661 raft_consensus.cc:2258] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.533447 28653 ts_tablet_manager.cc:1875] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.534641 28664 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.534696 28664 ts_tablet_manager.cc:1367] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.534863 28664 raft_consensus.cc:348] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.534953 28664 raft_consensus.cc:374] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.534982 28664 raft_consensus.cc:729] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.535001 28653 ts_tablet_manager.cc:1888] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.535024 28664 consensus_queue.cc:260] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.535054 28653 log.cc:1196] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/50566b0702db4f85b581931f415ca42d
I20240503 02:09:18.535144 28664 ts_tablet_manager.cc:1398] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.535215 28664 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.535245 28653 ts_tablet_manager.cc:1909] T 50566b0702db4f85b581931f415ca42d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.535552 28664 tablet_bootstrap.cc:654] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.535671 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 50566b0702db4f85b581931f415ca42d (table default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]) successfully deleted
I20240503 02:09:18.535698 28766 raft_consensus.cc:482] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.535743 28766 raft_consensus.cc:504] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.535851 28766 leader_election.cc:288] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.535974 28661 ts_tablet_manager.cc:1875] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.536064 28664 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.536093 28664 ts_tablet_manager.cc:1367] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.536214 28664 raft_consensus.cc:348] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.536260 28664 raft_consensus.cc:374] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.536298 28664 raft_consensus.cc:729] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.536334 28664 consensus_queue.cc:260] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.536449 28664 ts_tablet_manager.cc:1398] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.536470 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec2373bab15403cbcae47fd20f3c7ce" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.536532 27718 raft_consensus.cc:2454] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.536669 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 03645bdcd1dc481f8d404544403c2f22 (DEFAULT_TABLE table=default.Table_51f3d037d2e4460fa3a131fc639d7eb3 [id=061292b5d8974e6ebec546d6ad47d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.536760 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03645bdcd1dc481f8d404544403c2f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.536803 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 03645bdcd1dc481f8d404544403c2f22 (DEFAULT_TABLE table=default.Table_51f3d037d2e4460fa3a131fc639d7eb3 [id=061292b5d8974e6ebec546d6ad47d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.536899 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03645bdcd1dc481f8d404544403c2f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.537405 27427 tablet_service.cc:1467] Processing CreateTablet for tablet e24bcff4301b4e6d972211654cb90cb5 (DEFAULT_TABLE table=default.Table_d7d1d67ae02d4f349e3c19d354ad1f74 [id=6263fdbb7ede46dcabb3e5b55ddca5fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.537488 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24bcff4301b4e6d972211654cb90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.537541 27388 leader_election.cc:302] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.537693 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 03645bdcd1dc481f8d404544403c2f22 (DEFAULT_TABLE table=default.Table_51f3d037d2e4460fa3a131fc639d7eb3 [id=061292b5d8974e6ebec546d6ad47d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.537719 27560 tablet_service.cc:1467] Processing CreateTablet for tablet e24bcff4301b4e6d972211654cb90cb5 (DEFAULT_TABLE table=default.Table_d7d1d67ae02d4f349e3c19d354ad1f74 [id=6263fdbb7ede46dcabb3e5b55ddca5fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.537789 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24bcff4301b4e6d972211654cb90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.537868 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03645bdcd1dc481f8d404544403c2f22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.538002 27703 tablet_service.cc:1467] Processing CreateTablet for tablet e24bcff4301b4e6d972211654cb90cb5 (DEFAULT_TABLE table=default.Table_d7d1d67ae02d4f349e3c19d354ad1f74 [id=6263fdbb7ede46dcabb3e5b55ddca5fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.538101 28655 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.538491 28655 tablet_bootstrap.cc:654] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.539012 28654 ts_tablet_manager.cc:1875] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.539136 29550 raft_consensus.cc:2781] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.539175 29550 raft_consensus.cc:482] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.539194 29550 raft_consensus.cc:3037] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.539394 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec2373bab15403cbcae47fd20f3c7ce" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.539455 27582 raft_consensus.cc:2454] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.539803 29550 raft_consensus.cc:504] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.539904 29550 leader_election.cc:288] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.540210 28654 ts_tablet_manager.cc:1888] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.540251 28654 log.cc:1196] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/50566b0702db4f85b581931f415ca42d
I20240503 02:09:18.540385 28654 ts_tablet_manager.cc:1909] T 50566b0702db4f85b581931f415ca42d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.540418 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec2373bab15403cbcae47fd20f3c7ce" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.540480 27582 raft_consensus.cc:3037] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.540753 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec2373bab15403cbcae47fd20f3c7ce" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.540827 27718 raft_consensus.cc:3037] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.541121 27582 raft_consensus.cc:2454] T 7ec2373bab15403cbcae47fd20f3c7ce P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.541389 27386 leader_election.cc:302] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.541467 28766 raft_consensus.cc:2781] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.541522 28766 raft_consensus.cc:686] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.541539 27718 raft_consensus.cc:2454] T 7ec2373bab15403cbcae47fd20f3c7ce P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.541568 28766 consensus_queue.cc:237] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.542131 28656 tablet_bootstrap.cc:492] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.542176 28656 ts_tablet_manager.cc:1367] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.001s
I20240503 02:09:18.542308 28656 raft_consensus.cc:348] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.542371 28656 raft_consensus.cc:374] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.542388 28656 raft_consensus.cc:729] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.542438 28656 consensus_queue.cc:260] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.542517 28661 ts_tablet_manager.cc:1888] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.542546 28656 ts_tablet_manager.cc:1398] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.542560 28661 log.cc:1196] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/50566b0702db4f85b581931f415ca42d
I20240503 02:09:18.542622 28656 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.542737 28661 ts_tablet_manager.cc:1909] T 50566b0702db4f85b581931f415ca42d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.543009 28656 tablet_bootstrap.cc:654] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.543048 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e24bcff4301b4e6d972211654cb90cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.543659 28664 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.543969 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 50566b0702db4f85b581931f415ca42d (table default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]) successfully deleted
I20240503 02:09:18.544112 28664 tablet_bootstrap.cc:654] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.544857 27200 catalog_manager.cc:5484] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.545011 28664 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.545045 28664 ts_tablet_manager.cc:1367] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.545173 28664 raft_consensus.cc:348] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.545231 28664 raft_consensus.cc:374] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.545248 28664 raft_consensus.cc:729] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.545289 28664 consensus_queue.cc:260] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.545727 28664 ts_tablet_manager.cc:1398] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.545797 28664 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.546020 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1d6290a74d50458d9837eb9219ab0aa9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.546133 28664 tablet_bootstrap.cc:654] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:18.546133 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d6290a74d50458d9837eb9219ab0aa9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.546970 27453 raft_consensus.cc:1261] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.547116 27582 raft_consensus.cc:1261] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.547194 28766 raft_consensus.cc:482] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.547231 28766 raft_consensus.cc:504] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.547333 28766 leader_election.cc:288] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.547484 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 50566b0702db4f85b581931f415ca42d (table default.Table_ec955a8689124c99a40190db45718286 [id=d596f31dfc2d41ee9478ab2c02829796]) successfully deleted
I20240503 02:09:18.547632 29643 consensus_queue.cc:1035] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.547807 28655 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.547849 28655 ts_tablet_manager.cc:1367] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240503 02:09:18.548002 28655 raft_consensus.cc:348] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.548075 28655 raft_consensus.cc:374] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.548095 28655 raft_consensus.cc:729] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.548112 28774 consensus_queue.cc:1035] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.548137 28655 consensus_queue.cc:260] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.548236 28655 ts_tablet_manager.cc:1398] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.548339 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcf1601f3ea485ca4ad4a58cf0c7ee7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.548400 27722 raft_consensus.cc:2454] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.548415 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcf1601f3ea485ca4ad4a58cf0c7ee7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.548463 27582 raft_consensus.cc:2454] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.548666 27386 leader_election.cc:302] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.548729 28766 raft_consensus.cc:2781] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.548750 28766 raft_consensus.cc:482] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.548766 28766 raft_consensus.cc:3037] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.548969 28664 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.549003 28664 ts_tablet_manager.cc:1367] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.549119 28664 raft_consensus.cc:348] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.549165 28664 raft_consensus.cc:374] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.549180 28664 raft_consensus.cc:729] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.549219 28664 consensus_queue.cc:260] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.549428 28766 raft_consensus.cc:504] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.549517 28766 leader_election.cc:288] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.549306 28664 ts_tablet_manager.cc:1398] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.549690 29643 raft_consensus.cc:482] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.550241 29643 raft_consensus.cc:504] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.550330 29643 leader_election.cc:288] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.550325 28655 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.550691 28655 tablet_bootstrap.cc:654] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.550696 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcf1601f3ea485ca4ad4a58cf0c7ee7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.550753 27718 raft_consensus.cc:3037] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.550971 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60278e574634b7aaba6ef4412dd2dba" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.551028 27587 raft_consensus.cc:2454] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.551054 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.551117 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dcf1601f3ea485ca4ad4a58cf0c7ee7" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.551148 27586 raft_consensus.cc:3037] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.551308 27656 leader_election.cc:302] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.551383 27718 raft_consensus.cc:2454] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.551429 28740 raft_consensus.cc:2781] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.551468 28740 raft_consensus.cc:482] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.551486 28740 raft_consensus.cc:3037] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.551807 27586 raft_consensus.cc:2454] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.552126 28740 raft_consensus.cc:504] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.552218 28740 leader_election.cc:288] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.552248 28740 consensus_queue.cc:1035] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.552315 28740 raft_consensus.cc:482] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.552341 28740 raft_consensus.cc:504] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.552410 28740 leader_election.cc:288] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.552668 27197 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_0bcd732b52b74ae6a8101e4925faada8" } new_table_name: "default.Table_d9068f39301142349a144e840d90f268" modify_external_catalogs: true
I20240503 02:09:18.553012 27453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60278e574634b7aaba6ef4412dd2dba" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.553042 27451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60278e574634b7aaba6ef4412dd2dba" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.553078 27453 raft_consensus.cc:2454] T b60278e574634b7aaba6ef4412dd2dba P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.553153 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0e3d17da1d4d189ce0fdbf5725e289" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.553174 27388 leader_election.cc:302] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.553190 27452 raft_consensus.cc:2454] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.553233 29550 raft_consensus.cc:2781] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.553272 29550 raft_consensus.cc:686] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.553305 29550 consensus_queue.cc:237] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.553678 28507 raft_consensus.cc:482] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.553722 28507 raft_consensus.cc:504] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.553822 28507 leader_election.cc:288] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.553846 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0e3d17da1d4d189ce0fdbf5725e289" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.553905 27587 raft_consensus.cc:2454] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.554044 27659 leader_election.cc:302] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.554112 29643 raft_consensus.cc:2781] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.554136 29643 raft_consensus.cc:482] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.554152 29643 raft_consensus.cc:3037] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.554756 29643 raft_consensus.cc:504] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.554876 29643 leader_election.cc:288] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.555335 27560 tablet_service.cc:1467] Processing CreateTablet for tablet ade5c5386bb849a39ecd95453a4ea147 (DEFAULT_TABLE table=default.Table_1d6290a74d50458d9837eb9219ab0aa9 [id=89755c3012e942adb4689ab658d85e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.555433 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ade5c5386bb849a39ecd95453a4ea147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.555562 27427 tablet_service.cc:1467] Processing CreateTablet for tablet ade5c5386bb849a39ecd95453a4ea147 (DEFAULT_TABLE table=default.Table_1d6290a74d50458d9837eb9219ab0aa9 [id=89755c3012e942adb4689ab658d85e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.555656 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ade5c5386bb849a39ecd95453a4ea147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.556316 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0e3d17da1d4d189ce0fdbf5725e289" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.556347 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e0cc2ff5594a28af52d056c6d6942d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.556388 27452 raft_consensus.cc:3037] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.556404 27718 raft_consensus.cc:2454] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.556883 27523 leader_election.cc:302] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.556921 27452 raft_consensus.cc:2454] T ea0e3d17da1d4d189ce0fdbf5725e289 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.556957 28507 raft_consensus.cc:2781] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.556982 28507 raft_consensus.cc:482] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.556996 28507 raft_consensus.cc:3037] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.557145 27702 tablet_service.cc:1467] Processing CreateTablet for tablet ade5c5386bb849a39ecd95453a4ea147 (DEFAULT_TABLE table=default.Table_1d6290a74d50458d9837eb9219ab0aa9 [id=89755c3012e942adb4689ab658d85e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.557176 28655 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.557224 28655 ts_tablet_manager.cc:1367] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.557238 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ade5c5386bb849a39ecd95453a4ea147. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.557381 28655 raft_consensus.cc:348] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.557433 28655 raft_consensus.cc:374] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.557449 28655 raft_consensus.cc:729] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.557485 28655 consensus_queue.cc:260] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.557448 27195 catalog_manager.cc:5484] T 0dcf1601f3ea485ca4ad4a58cf0c7ee7 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.557544 27195 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.557581 28655 ts_tablet_manager.cc:1398] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.557633 28655 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.557651 28507 raft_consensus.cc:504] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.557750 28507 leader_election.cc:288] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.558014 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8b859087b6e64ad6ac2a7d426f69e6d1" } new_table_name: "default.Table_9365fa9ad6e04796b837978abd0f4d3e" modify_external_catalogs: true
I20240503 02:09:18.558089 28655 tablet_bootstrap.cc:654] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.558178 27659 leader_election.cc:302] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.558248 29643 raft_consensus.cc:2781] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.558282 29643 raft_consensus.cc:686] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.558315 29643 consensus_queue.cc:237] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.558480 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e0cc2ff5594a28af52d056c6d6942d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.558538 27718 raft_consensus.cc:3037] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.559151 27718 raft_consensus.cc:2454] T 16e0cc2ff5594a28af52d056c6d6942d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.559284 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea0e3d17da1d4d189ce0fdbf5725e289" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.559345 27582 raft_consensus.cc:3037] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.559603 27523 leader_election.cc:302] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.559995 27582 raft_consensus.cc:2454] T ea0e3d17da1d4d189ce0fdbf5725e289 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.560122 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e0cc2ff5594a28af52d056c6d6942d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.560181 27450 raft_consensus.cc:3037] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.560362 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60278e574634b7aaba6ef4412dd2dba" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.560433 27586 raft_consensus.cc:3037] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.560715 27450 raft_consensus.cc:2454] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.560968 27586 raft_consensus.cc:2454] T b60278e574634b7aaba6ef4412dd2dba P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.558535 28664 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.561126 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16e0cc2ff5594a28af52d056c6d6942d" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.561178 27452 raft_consensus.cc:2362] T 16e0cc2ff5594a28af52d056c6d6942d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:18.561235 28507 raft_consensus.cc:2781] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.561270 28507 raft_consensus.cc:686] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.561306 28507 consensus_queue.cc:237] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.561493 28664 tablet_bootstrap.cc:654] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.562794 28655 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.562841 28655 ts_tablet_manager.cc:1367] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.562969 28655 raft_consensus.cc:348] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.563030 28655 raft_consensus.cc:374] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.563051 28655 raft_consensus.cc:729] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.563091 28655 consensus_queue.cc:260] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.563181 28655 ts_tablet_manager.cc:1398] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.563478 28664 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.563530 28664 ts_tablet_manager.cc:1367] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.563655 28664 raft_consensus.cc:348] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.563715 28664 raft_consensus.cc:374] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.563732 28664 raft_consensus.cc:729] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.563767 28664 consensus_queue.cc:260] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.563848 28664 ts_tablet_manager.cc:1398] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.566174 27197 catalog_manager.cc:5484] T 16e0cc2ff5594a28af52d056c6d6942d P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.568539 27656 leader_election.cc:302] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.568673 28774 raft_consensus.cc:2781] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.568703 30370 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.568734 28774 raft_consensus.cc:686] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.568776 28774 consensus_queue.cc:237] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.568843 29643 consensus_queue.cc:1035] T 26ff4d9b25b546ee99946ccc34c22669 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.569370 30190 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.569836 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.569985 27452 raft_consensus.cc:1261] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.570050 30170 tablet.cc:1722] T 82e659f5818744f584e64997789ebf86 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20240503 02:09:18.570290 28693 consensus_queue.cc:1035] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.572299 27200 catalog_manager.cc:5484] T b60278e574634b7aaba6ef4412dd2dba P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.572414 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 7b820e0b556240b393ee31f0eae3a36b (table default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b]). Expected version 1 got 0
I20240503 02:09:18.572435 27582 raft_consensus.cc:1261] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.572451 27200 catalog_manager.cc:5484] T ea0e3d17da1d4d189ce0fdbf5725e289 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.572644 28740 consensus_queue.cc:1035] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.573109 30370 tablet.cc:1722] T 7b820e0b556240b393ee31f0eae3a36b P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.573932 30190 tablet.cc:1722] T 78330603666e4171845cf14dade17795 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.574322 28656 tablet_bootstrap.cc:492] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.574364 28656 ts_tablet_manager.cc:1367] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.032s user 0.000s sys 0.000s
I20240503 02:09:18.574527 28656 raft_consensus.cc:348] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.574584 28656 raft_consensus.cc:374] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.574601 28656 raft_consensus.cc:729] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.574637 28656 consensus_queue.cc:260] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.574739 28656 ts_tablet_manager.cc:1398] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.575500 29558 tablet.cc:1722] T 78330603666e4171845cf14dade17795 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.575758 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3b492931b6ca49b096365669e265e21d" } new_table_name: "default.Table_3a6a4ea4251f4e209b42695519fe0f47" modify_external_catalogs: true
I20240503 02:09:18.575883 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_d9068f39301142349a144e840d90f268 [id=c10aff1a98f64f13a7ffb57bf8a6e815] alter complete (version 3)
I20240503 02:09:18.576840 30364 tablet.cc:1722] T a551225a4c8f4e4f84410bcb6fa38634 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.577617 30365 tablet.cc:1722] T 7b820e0b556240b393ee31f0eae3a36b P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.578300 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet a551225a4c8f4e4f84410bcb6fa38634 (table default.Table_9365fa9ad6e04796b837978abd0f4d3e [id=e9d7bbda984d4b358f204abf5ac862f0]). Expected version 1 got 0
I20240503 02:09:18.579105 30170 tablet.cc:1722] T 78330603666e4171845cf14dade17795 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.579304 28710 tablet.cc:1722] T a551225a4c8f4e4f84410bcb6fa38634 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.580128 28766 raft_consensus.cc:482] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.580179 28766 raft_consensus.cc:504] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.580278 28766 leader_election.cc:288] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.580796 28656 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.580809 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24bcff4301b4e6d972211654cb90cb5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.580958 30190 tablet.cc:1722] T 7b820e0b556240b393ee31f0eae3a36b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.581334 28656 tablet_bootstrap.cc:654] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:18.581526 27386 leader_election.cc:341] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.581972 28656 tablet_bootstrap.cc:492] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.582020 28656 ts_tablet_manager.cc:1367] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.582151 28656 raft_consensus.cc:348] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.582211 28656 raft_consensus.cc:374] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.582231 28656 raft_consensus.cc:729] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.582266 28656 consensus_queue.cc:260] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.582379 28656 ts_tablet_manager.cc:1398] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.582435 28656 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.582825 28656 tablet_bootstrap.cc:654] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.583202 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_9365fa9ad6e04796b837978abd0f4d3e [id=e9d7bbda984d4b358f204abf5ac862f0] alter complete (version 1)
I20240503 02:09:18.583726 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet a551225a4c8f4e4f84410bcb6fa38634 (table default.Table_9365fa9ad6e04796b837978abd0f4d3e [id=e9d7bbda984d4b358f204abf5ac862f0]). Expected version 1 got 0
I20240503 02:09:18.583894 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24bcff4301b4e6d972211654cb90cb5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.583962 27722 raft_consensus.cc:2454] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.584231 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9019f5ee81ad45399f17053162dc0be8" } modify_external_catalogs: true
I20240503 02:09:18.584290 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9019f5ee81ad45399f17053162dc0be8" } modify_external_catalogs: true
I20240503 02:09:18.584290 27388 leader_election.cc:302] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db
I20240503 02:09:18.584393 28766 raft_consensus.cc:2781] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.584424 28766 raft_consensus.cc:482] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.584439 28766 raft_consensus.cc:3037] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.584760 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_2248635d13b248b3a0d8a63ce5e8a324 [id=988d19150c7148f19648ef6a7eea345b] alter complete (version 1)
I20240503 02:09:18.585068 28766 raft_consensus.cc:504] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.585177 28766 leader_election.cc:288] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.585199 28656 tablet_bootstrap.cc:492] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.585244 28656 ts_tablet_manager.cc:1367] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.585400 28656 raft_consensus.cc:348] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.585458 28656 raft_consensus.cc:374] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.585474 28656 raft_consensus.cc:729] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.585502 28656 consensus_queue.cc:260] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.585594 28656 ts_tablet_manager.cc:1398] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.585636 28656 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.585964 28656 tablet_bootstrap.cc:654] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.586272 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24bcff4301b4e6d972211654cb90cb5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.586333 27585 raft_consensus.cc:3037] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.586779 28656 tablet_bootstrap.cc:492] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.586819 28656 ts_tablet_manager.cc:1367] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.586951 27585 raft_consensus.cc:2454] T e24bcff4301b4e6d972211654cb90cb5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.586956 28656 raft_consensus.cc:348] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.587044 28656 raft_consensus.cc:374] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.587061 28656 raft_consensus.cc:729] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.587113 28656 consensus_queue.cc:260] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.587216 28656 ts_tablet_manager.cc:1398] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.587462 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_bdb8cc2c2da4465981bf233f470eff28 [id=b743de8e555f4387b6ec8bd798231c72] alter complete (version 1)
I20240503 02:09:18.587594 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 138ffe8057904a2485289b61064bf382
I20240503 02:09:18.587637 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e24bcff4301b4e6d972211654cb90cb5" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.587688 27722 raft_consensus.cc:3037] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.586983 30190 tablet.cc:1722] T a551225a4c8f4e4f84410bcb6fa38634 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.587910 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffe8057904a2485289b61064bf382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.588268 28661 tablet_replica.cc:330] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.588317 27722 raft_consensus.cc:2454] T e24bcff4301b4e6d972211654cb90cb5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.588335 28661 raft_consensus.cc:2229] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.588405 28661 raft_consensus.cc:2258] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.588665 27388 leader_election.cc:302] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.588737 28766 raft_consensus.cc:2781] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.588775 28766 raft_consensus.cc:686] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.588814 28766 consensus_queue.cc:237] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.588910 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffe8057904a2485289b61064bf382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.588948 28410 raft_consensus.cc:482] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.588994 28410 raft_consensus.cc:504] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.589083 28410 leader_election.cc:288] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:18.589233 28653 tablet_replica.cc:330] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.589299 28653 raft_consensus.cc:2229] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.589284 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 138ffe8057904a2485289b61064bf382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.589359 28653 raft_consensus.cc:2258] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.589426 28654 tablet_replica.cc:330] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.589480 28654 raft_consensus.cc:2229] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.589478 28661 ts_tablet_manager.cc:1875] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.589522 28654 raft_consensus.cc:2258] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.589757 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04682e3c79e4f8ea8f045a83f785da6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.589820 27450 raft_consensus.cc:2454] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.590116 27524 leader_election.cc:302] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.590252 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6cffc813b4ed451a91ad4fa95b35f4f3" } modify_external_catalogs: true
I20240503 02:09:18.590307 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6cffc813b4ed451a91ad4fa95b35f4f3" } modify_external_catalogs: true
I20240503 02:09:18.590342 28654 ts_tablet_manager.cc:1875] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.590497 27195 catalog_manager.cc:5484] T e24bcff4301b4e6d972211654cb90cb5 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.590795 28661 ts_tablet_manager.cc:1888] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.590838 28661 log.cc:1196] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/138ffe8057904a2485289b61064bf382
I20240503 02:09:18.591020 28661 ts_tablet_manager.cc:1909] T 138ffe8057904a2485289b61064bf382 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.591387 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 138ffe8057904a2485289b61064bf382 (table default.Table_9019f5ee81ad45399f17053162dc0be8 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:18.591682 28654 ts_tablet_manager.cc:1888] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.591730 28654 log.cc:1196] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/138ffe8057904a2485289b61064bf382
I20240503 02:09:18.591682 29241 raft_consensus.cc:2781] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.591802 29241 raft_consensus.cc:482] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.591820 29241 raft_consensus.cc:3037] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.591917 28654 ts_tablet_manager.cc:1909] T 138ffe8057904a2485289b61064bf382 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.591979 30009 tablet.cc:1722] T fa67973d02924925a35b4418e1d4fa29 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.592267 30370 tablet.cc:1722] T fa67973d02924925a35b4418e1d4fa29 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.592303 29550 raft_consensus.cc:482] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.592340 29550 raft_consensus.cc:504] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.592430 29550 leader_election.cc:288] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.592497 29241 raft_consensus.cc:504] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.592586 29241 leader_election.cc:288] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597)
I20240503 02:09:18.592587 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ade5c5386bb849a39ecd95453a4ea147" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.592645 27718 raft_consensus.cc:2454] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.592705 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ade5c5386bb849a39ecd95453a4ea147" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.592752 27582 raft_consensus.cc:2454] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.592809 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04682e3c79e4f8ea8f045a83f785da6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.592851 27450 raft_consensus.cc:3037] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.592890 28653 ts_tablet_manager.cc:1875] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.593127 27386 leader_election.cc:302] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.593192 29550 raft_consensus.cc:2781] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.593214 29550 raft_consensus.cc:482] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.593230 29550 raft_consensus.cc:3037] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.593359 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 138ffe8057904a2485289b61064bf382 (table default.Table_9019f5ee81ad45399f17053162dc0be8 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:18.593482 27450 raft_consensus.cc:2454] T e04682e3c79e4f8ea8f045a83f785da6 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.593786 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04682e3c79e4f8ea8f045a83f785da6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.593842 27718 raft_consensus.cc:3037] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.594143 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 1ed2da494a334cbab2f24ca578adc494
I20240503 02:09:18.594233 27524 leader_election.cc:302] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.594292 28653 ts_tablet_manager.cc:1888] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.594314 28507 raft_consensus.cc:2781] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.594334 28653 log.cc:1196] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/138ffe8057904a2485289b61064bf382
I20240503 02:09:18.594347 28507 raft_consensus.cc:686] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.594383 28507 consensus_queue.cc:237] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.594465 27718 raft_consensus.cc:2454] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.594517 28653 ts_tablet_manager.cc:1909] T 138ffe8057904a2485289b61064bf382 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.594573 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed2da494a334cbab2f24ca578adc494 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.594705 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed2da494a334cbab2f24ca578adc494 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.594861 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed2da494a334cbab2f24ca578adc494 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.595129 29550 raft_consensus.cc:504] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.595252 29550 leader_election.cc:288] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.595360 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ade5c5386bb849a39ecd95453a4ea147" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.595409 27582 raft_consensus.cc:3037] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.596001 28654 tablet_replica.cc:330] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.595976 27198 catalog_manager.cc:5484] T e04682e3c79e4f8ea8f045a83f785da6 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.596047 27582 raft_consensus.cc:2454] T ade5c5386bb849a39ecd95453a4ea147 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.596058 28654 raft_consensus.cc:2229] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.596117 28654 raft_consensus.cc:2258] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.596200 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e04682e3c79e4f8ea8f045a83f785da6" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.596259 27722 raft_consensus.cc:2362] T e04682e3c79e4f8ea8f045a83f785da6 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1. Re-sending same reply.
I20240503 02:09:18.596338 27386 leader_election.cc:302] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.596766 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ade5c5386bb849a39ecd95453a4ea147" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.596822 27718 raft_consensus.cc:3037] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.597151 28653 tablet_replica.cc:330] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.597216 28653 raft_consensus.cc:2229] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.597245 28653 raft_consensus.cc:2258] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.597417 27718 raft_consensus.cc:2454] T ade5c5386bb849a39ecd95453a4ea147 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.597585 28661 tablet_replica.cc:330] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.597647 28661 raft_consensus.cc:2229] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.597682 28661 raft_consensus.cc:2258] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.597802 28766 raft_consensus.cc:2781] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.597863 28766 raft_consensus.cc:686] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.597901 28766 consensus_queue.cc:237] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.597939 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 138ffe8057904a2485289b61064bf382 (table default.Table_9019f5ee81ad45399f17053162dc0be8 [id=cd9e7541be36427ea8e01bbba147d869]) successfully deleted
I20240503 02:09:18.598006 28661 ts_tablet_manager.cc:1875] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.598899 28710 tablet.cc:1722] T fa67973d02924925a35b4418e1d4fa29 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.599218 28653 ts_tablet_manager.cc:1875] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.600373 27853 consensus_queue.cc:1035] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.600479 28653 ts_tablet_manager.cc:1888] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.600522 28653 log.cc:1196] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/1ed2da494a334cbab2f24ca578adc494
I20240503 02:09:18.600689 28653 ts_tablet_manager.cc:1909] T 1ed2da494a334cbab2f24ca578adc494 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.601210 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 25063c03d7dd4e7a87b7e8bd315684a8 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.601544 27200 catalog_manager.cc:5484] T ade5c5386bb849a39ecd95453a4ea147 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.601758 28661 ts_tablet_manager.cc:1888] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.601800 28661 log.cc:1196] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/1ed2da494a334cbab2f24ca578adc494
I20240503 02:09:18.601987 28661 ts_tablet_manager.cc:1909] T 1ed2da494a334cbab2f24ca578adc494 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.603674 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 1ed2da494a334cbab2f24ca578adc494 (table default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]) successfully deleted
I20240503 02:09:18.603950 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 1ed2da494a334cbab2f24ca578adc494 (table default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]) successfully deleted
I20240503 02:09:18.604874 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 25063c03d7dd4e7a87b7e8bd315684a8
I20240503 02:09:18.604990 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 25063c03d7dd4e7a87b7e8bd315684a8 deleted and replaced by tablet 8bec77949895463791227ccf6e309f27
I20240503 02:09:18.605190 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8bec77949895463791227ccf6e309f27) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.605628 27853 consensus_queue.cc:1035] T 633faa7ea971417dbb6116cb7dcb3ef3 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.605691 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8bec77949895463791227ccf6e309f27) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.605823 29550 consensus_queue.cc:1035] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.606089 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 25063c03d7dd4e7a87b7e8bd315684a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8bec77949895463791227ccf6e309f27) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.606627 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_3a6a4ea4251f4e209b42695519fe0f47 [id=14ac8b9e93234f449dcf71726c9b7a63] alter complete (version 1)
I20240503 02:09:18.607074 28653 tablet_replica.cc:330] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.607197 28653 raft_consensus.cc:2229] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.607679 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 8bec77949895463791227ccf6e309f27 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.607815 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bec77949895463791227ccf6e309f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.607764 28653 raft_consensus.cc:2258] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.607829 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 8bec77949895463791227ccf6e309f27 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.605870 28661 tablet_replica.cc:330] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.608434 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 8bec77949895463791227ccf6e309f27 (DEFAULT_TABLE table=default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.608530 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bec77949895463791227ccf6e309f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.609401 28656 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.609640 28654 ts_tablet_manager.cc:1875] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.609933 28656 tablet_bootstrap.cc:654] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.610034 28655 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.610407 28655 tablet_bootstrap.cc:654] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.610793 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bec77949895463791227ccf6e309f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.610971 28653 ts_tablet_manager.cc:1875] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.611088 29551 consensus_queue.cc:1035] T c934fb82221c45bab114ff8ac97da2bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.611399 28654 ts_tablet_manager.cc:1888] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.611536 28654 log.cc:1196] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/1ed2da494a334cbab2f24ca578adc494
I20240503 02:09:18.611732 28654 ts_tablet_manager.cc:1909] T 1ed2da494a334cbab2f24ca578adc494 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.611945 28654 tablet_replica.cc:330] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.612010 28654 raft_consensus.cc:2229] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.612066 28654 raft_consensus.cc:2258] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.612039 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 1ed2da494a334cbab2f24ca578adc494 (table default.Table_6cffc813b4ed451a91ad4fa95b35f4f3 [id=b99cea6f0f2e445b902e7663da48d344]) successfully deleted
I20240503 02:09:18.612252 28664 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.612669 28664 tablet_bootstrap.cc:654] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.612902 28661 raft_consensus.cc:2229] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.613014 28661 raft_consensus.cc:2258] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.612948 28653 ts_tablet_manager.cc:1888] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.613166 28653 log.cc:1196] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/25063c03d7dd4e7a87b7e8bd315684a8
I20240503 02:09:18.613428 28653 ts_tablet_manager.cc:1909] T 25063c03d7dd4e7a87b7e8bd315684a8 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.613801 27853 consensus_queue.cc:1035] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.614410 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 25063c03d7dd4e7a87b7e8bd315684a8 (table default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]) successfully deleted
W20240503 02:09:18.614733 27198 catalog_manager.cc:5317] Ignoring report from unknown tablet 85e1f031e13f47c692de9a8a5e256613
I20240503 02:09:18.613754 28661 ts_tablet_manager.cc:1875] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.616155 28654 ts_tablet_manager.cc:1875] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.617239 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 7a8f3809378f444386cdd868e6db4c04 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.617364 28654 ts_tablet_manager.cc:1888] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.617460 28654 log.cc:1196] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/25063c03d7dd4e7a87b7e8bd315684a8
I20240503 02:09:18.617972 28661 ts_tablet_manager.cc:1888] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.618024 28661 log.cc:1196] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/25063c03d7dd4e7a87b7e8bd315684a8
I20240503 02:09:18.618201 28654 ts_tablet_manager.cc:1909] T 25063c03d7dd4e7a87b7e8bd315684a8 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.618260 28661 ts_tablet_manager.cc:1909] T 25063c03d7dd4e7a87b7e8bd315684a8 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.618543 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 25063c03d7dd4e7a87b7e8bd315684a8 (table default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]) successfully deleted
I20240503 02:09:18.618638 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 7a8f3809378f444386cdd868e6db4c04
I20240503 02:09:18.618713 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 7a8f3809378f444386cdd868e6db4c04 deleted and replaced by tablet 9201a0e5862b41878733a6f690600c66
I20240503 02:09:18.618921 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a8f3809378f444386cdd868e6db4c04 with delete_type TABLET_DATA_DELETED (replaced by tablet 9201a0e5862b41878733a6f690600c66) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.619028 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a8f3809378f444386cdd868e6db4c04 with delete_type TABLET_DATA_DELETED (replaced by tablet 9201a0e5862b41878733a6f690600c66) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.619206 28661 tablet_replica.cc:330] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.619203 28653 tablet_replica.cc:330] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.619266 28661 raft_consensus.cc:2229] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.619264 28653 raft_consensus.cc:2229] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.619297 28653 raft_consensus.cc:2258] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.619297 28661 raft_consensus.cc:2258] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.619405 28653 ts_tablet_manager.cc:1875] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.619441 28661 ts_tablet_manager.cc:1875] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.619661 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a8f3809378f444386cdd868e6db4c04 with delete_type TABLET_DATA_DELETED (replaced by tablet 9201a0e5862b41878733a6f690600c66) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.619984 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 25063c03d7dd4e7a87b7e8bd315684a8 (table default.Table_ee10c86dfb9c4459b3cdc8fd11c48bad [id=385bfee284bd452e9caac1cc111f1c0c]) successfully deleted
I20240503 02:09:18.620105 28654 tablet_replica.cc:330] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.620203 28654 raft_consensus.cc:2229] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.620304 28654 raft_consensus.cc:2258] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.620456 28654 ts_tablet_manager.cc:1875] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.620743 28653 ts_tablet_manager.cc:1888] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.620743 28661 ts_tablet_manager.cc:1888] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.621152 28653 log.cc:1196] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/7a8f3809378f444386cdd868e6db4c04
I20240503 02:09:18.621312 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 9201a0e5862b41878733a6f690600c66 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.621533 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201a0e5862b41878733a6f690600c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.621444 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 9201a0e5862b41878733a6f690600c66 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.621873 27853 consensus_queue.cc:1035] T c6179ea574834eeb94ae97cbc8cc87c5 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.622146 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201a0e5862b41878733a6f690600c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.621842 28410 raft_consensus.cc:482] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.623337 28653 ts_tablet_manager.cc:1909] T 7a8f3809378f444386cdd868e6db4c04 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.623356 28410 raft_consensus.cc:504] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.623467 28410 leader_election.cc:288] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.623603 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03645bdcd1dc481f8d404544403c2f22" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.623615 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03645bdcd1dc481f8d404544403c2f22" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.623674 27718 raft_consensus.cc:2454] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.623672 27447 raft_consensus.cc:2454] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.623842 27523 leader_election.cc:302] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.622020 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 9201a0e5862b41878733a6f690600c66 (DEFAULT_TABLE table=default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.623991 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9201a0e5862b41878733a6f690600c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.624075 28410 raft_consensus.cc:2781] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.624114 28410 raft_consensus.cc:482] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.624132 28410 raft_consensus.cc:3037] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.624366 28656 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.624404 28656 ts_tablet_manager.cc:1367] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20240503 02:09:18.624544 28656 raft_consensus.cc:348] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.624598 28656 raft_consensus.cc:374] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.624632 28656 raft_consensus.cc:729] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.624665 28656 consensus_queue.cc:260] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.624704 28410 raft_consensus.cc:504] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.624763 28656 ts_tablet_manager.cc:1398] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.624814 28656 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.624922 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03645bdcd1dc481f8d404544403c2f22" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.624976 27447 raft_consensus.cc:3037] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.625212 28656 tablet_bootstrap.cc:654] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.625574 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 7a8f3809378f444386cdd868e6db4c04 (table default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]) successfully deleted
I20240503 02:09:18.625576 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03645bdcd1dc481f8d404544403c2f22" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.625636 27718 raft_consensus.cc:3037] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.624815 28410 leader_election.cc:288] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.625650 27447 raft_consensus.cc:2454] T 03645bdcd1dc481f8d404544403c2f22 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.625972 27524 leader_election.cc:302] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.626051 28410 raft_consensus.cc:2781] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.626092 28410 raft_consensus.cc:686] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.626129 28410 consensus_queue.cc:237] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.626279 27718 raft_consensus.cc:2454] T 03645bdcd1dc481f8d404544403c2f22 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.626415 27853 consensus_queue.cc:1035] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.626610 28664 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.626660 28664 ts_tablet_manager.cc:1367] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:18.626801 28664 raft_consensus.cc:348] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.626852 28664 raft_consensus.cc:374] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.626868 28664 raft_consensus.cc:729] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.626910 28664 consensus_queue.cc:260] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.627000 28664 ts_tablet_manager.cc:1398] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.627060 28664 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.627414 28664 tablet_bootstrap.cc:654] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.629231 28774 consensus_queue.cc:1035] T a0e60cbc3e764f71bcfb0ae5d0558fd8 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.629976 27198 master_service.cc:946] ReplaceTablet: received request to replace tablet 609e5f3b76ac419ea61ae8a2ac42d22f from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.630529 28656 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.630573 28656 ts_tablet_manager.cc:1367] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.630676 28656 raft_consensus.cc:348] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.630731 28656 raft_consensus.cc:374] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.630749 28656 raft_consensus.cc:729] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.630666 27200 catalog_manager.cc:5484] T 03645bdcd1dc481f8d404544403c2f22 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.630785 28656 consensus_queue.cc:260] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.630884 28656 ts_tablet_manager.cc:1398] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.630961 28661 log.cc:1196] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/7a8f3809378f444386cdd868e6db4c04
I20240503 02:09:18.631322 28774 raft_consensus.cc:482] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.631373 28774 raft_consensus.cc:504] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.631465 28774 leader_election.cc:288] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.631677 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c502af17069460aa2ddfee4a9ffaed4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.631742 27447 raft_consensus.cc:2454] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.631899 27659 leader_election.cc:302] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.631973 28774 raft_consensus.cc:2781] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.632028 28774 raft_consensus.cc:482] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.632046 28774 raft_consensus.cc:3037] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.632067 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c502af17069460aa2ddfee4a9ffaed4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.632125 27582 raft_consensus.cc:2454] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.632393 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 609e5f3b76ac419ea61ae8a2ac42d22f
I20240503 02:09:18.632485 27198 catalog_manager.cc:6554] ReplaceTablet: tablet 609e5f3b76ac419ea61ae8a2ac42d22f deleted and replaced by tablet 1a7abb0f315e47cdaff61e8e5b22d1a2
I20240503 02:09:18.632726 28774 raft_consensus.cc:504] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.632761 29551 consensus_queue.cc:1035] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.632789 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a7abb0f315e47cdaff61e8e5b22d1a2) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.632826 28774 leader_election.cc:288] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.632884 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a7abb0f315e47cdaff61e8e5b22d1a2) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.632920 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 609e5f3b76ac419ea61ae8a2ac42d22f with delete_type TABLET_DATA_DELETED (replaced by tablet 1a7abb0f315e47cdaff61e8e5b22d1a2) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.633016 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c502af17069460aa2ddfee4a9ffaed4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.633044 28653 tablet_replica.cc:330] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.633085 27447 raft_consensus.cc:3037] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.633087 28661 ts_tablet_manager.cc:1909] T 7a8f3809378f444386cdd868e6db4c04 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.633126 28653 raft_consensus.cc:2229] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.633158 28653 raft_consensus.cc:2258] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.633292 28661 tablet_replica.cc:330] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.633358 28661 raft_consensus.cc:2229] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.633440 28661 raft_consensus.cc:2258] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.633481 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 7a8f3809378f444386cdd868e6db4c04 (table default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]) successfully deleted
I20240503 02:09:18.633700 28661 ts_tablet_manager.cc:1875] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.633778 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c502af17069460aa2ddfee4a9ffaed4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.633831 27587 raft_consensus.cc:3037] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.634320 27447 raft_consensus.cc:2454] T 8c502af17069460aa2ddfee4a9ffaed4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.634438 27587 raft_consensus.cc:2454] T 8c502af17069460aa2ddfee4a9ffaed4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.633705 28664 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.634538 28664 ts_tablet_manager.cc:1367] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.634696 28410 raft_consensus.cc:482] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.634696 28664 raft_consensus.cc:348] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.634749 28664 raft_consensus.cc:374] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.634737 28410 raft_consensus.cc:504] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.634766 28664 raft_consensus.cc:729] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.634812 28664 consensus_queue.cc:260] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.634833 28410 leader_election.cc:288] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.634891 27656 leader_election.cc:302] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.634903 28664 ts_tablet_manager.cc:1398] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.635581 28661 ts_tablet_manager.cc:1888] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.635622 28661 log.cc:1196] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/609e5f3b76ac419ea61ae8a2ac42d22f
I20240503 02:09:18.635718 28653 ts_tablet_manager.cc:1875] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.635797 28661 ts_tablet_manager.cc:1909] T 609e5f3b76ac419ea61ae8a2ac42d22f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.636281 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.636338 27718 raft_consensus.cc:2454] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.636466 27523 leader_election.cc:302] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.636809 27853 raft_consensus.cc:2781] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.636857 27853 raft_consensus.cc:686] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.636891 27853 consensus_queue.cc:237] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.636979 28653 ts_tablet_manager.cc:1888] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.637028 28653 log.cc:1196] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/609e5f3b76ac419ea61ae8a2ac42d22f
I20240503 02:09:18.637197 28653 ts_tablet_manager.cc:1909] T 609e5f3b76ac419ea61ae8a2ac42d22f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.637307 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 609e5f3b76ac419ea61ae8a2ac42d22f (table default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]) successfully deleted
I20240503 02:09:18.637459 28410 raft_consensus.cc:2781] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.637495 28410 raft_consensus.cc:482] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.637512 28410 raft_consensus.cc:3037] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.638101 28410 raft_consensus.cc:504] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.638209 28410 leader_election.cc:288] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.638355 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.638444 27718 raft_consensus.cc:3037] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.638674 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.638343 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.639048 27718 raft_consensus.cc:2454] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
W20240503 02:09:18.639204 27524 leader_election.cc:341] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:09:18.639274 27524 leader_election.cc:341] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.639328 27523 leader_election.cc:302] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.639402 28410 raft_consensus.cc:2781] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.639448 28410 raft_consensus.cc:686] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.639488 28410 consensus_queue.cc:237] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.639559 27195 catalog_manager.cc:5484] T 8c502af17069460aa2ddfee4a9ffaed4 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.639642 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 609e5f3b76ac419ea61ae8a2ac42d22f (table default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]) successfully deleted
I20240503 02:09:18.642226 27200 catalog_manager.cc:5484] T 9201a0e5862b41878733a6f690600c66 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.643056 29550 consensus_queue.cc:1035] T 5d79bb60500d483985daa3ab36380409 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.643432 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet f56dc81ff4644d18a0105722f2f2f1e7 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.644678 27201 master_service.cc:946] ReplaceTablet: received request to replace tablet 8b7a955c305c4ac3a6ecd9248943a618 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.647585 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet f56dc81ff4644d18a0105722f2f2f1e7
I20240503 02:09:18.647717 27201 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 8b7a955c305c4ac3a6ecd9248943a618
I20240503 02:09:18.649194 27197 catalog_manager.cc:6554] ReplaceTablet: tablet f56dc81ff4644d18a0105722f2f2f1e7 deleted and replaced by tablet 32314d265c0341d98f710e29f7786fbf
I20240503 02:09:18.649331 27201 catalog_manager.cc:6554] ReplaceTablet: tablet 8b7a955c305c4ac3a6ecd9248943a618 deleted and replaced by tablet 56fd56a81cf4467c937e670ee2703c47
I20240503 02:09:18.649628 27554 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 56fd56a81cf4467c937e670ee2703c47) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.649749 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 32314d265c0341d98f710e29f7786fbf) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.649787 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 32314d265c0341d98f710e29f7786fbf) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.649849 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 56fd56a81cf4467c937e670ee2703c47) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.649881 28653 tablet_replica.cc:330] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.649942 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7a955c305c4ac3a6ecd9248943a618 with delete_type TABLET_DATA_DELETED (replaced by tablet 56fd56a81cf4467c937e670ee2703c47) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.649953 28653 raft_consensus.cc:2229] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.650036 28653 raft_consensus.cc:2258] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.650312 28661 tablet_replica.cc:330] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.650372 28661 raft_consensus.cc:2229] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.650419 28661 raft_consensus.cc:2258] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.650439 28653 ts_tablet_manager.cc:1875] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.650475 28654 ts_tablet_manager.cc:1888] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.650519 28654 log.cc:1196] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/7a8f3809378f444386cdd868e6db4c04
I20240503 02:09:18.650599 28661 ts_tablet_manager.cc:1875] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.650698 28654 ts_tablet_manager.cc:1909] T 7a8f3809378f444386cdd868e6db4c04 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.650878 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet f56dc81ff4644d18a0105722f2f2f1e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 32314d265c0341d98f710e29f7786fbf) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.651723 28653 ts_tablet_manager.cc:1888] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.651757 28653 log.cc:1196] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/f56dc81ff4644d18a0105722f2f2f1e7
I20240503 02:09:18.651929 28661 ts_tablet_manager.cc:1888] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.651940 28653 ts_tablet_manager.cc:1909] T f56dc81ff4644d18a0105722f2f2f1e7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.651973 28661 log.cc:1196] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/f56dc81ff4644d18a0105722f2f2f1e7
I20240503 02:09:18.652170 28661 ts_tablet_manager.cc:1909] T f56dc81ff4644d18a0105722f2f2f1e7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.652488 28661 tablet_replica.cc:330] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.652559 28661 raft_consensus.cc:2229] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.652611 28661 raft_consensus.cc:2258] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.652592 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet f56dc81ff4644d18a0105722f2f2f1e7 (table default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]) successfully deleted
I20240503 02:09:18.652873 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8812ecdff34e469998f5419dcb07669b" } modify_external_catalogs: true
I20240503 02:09:18.652938 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8812ecdff34e469998f5419dcb07669b" } modify_external_catalogs: true
I20240503 02:09:18.653012 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 7a8f3809378f444386cdd868e6db4c04 (table default.Table_adb9ec2bd2f7474c89ea05c85a4ce91f [id=2cd717562b4a452695fcfaa422fc5fc8]) successfully deleted
I20240503 02:09:18.653095 28654 tablet_replica.cc:330] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.653371 28661 ts_tablet_manager.cc:1875] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.653460 28654 raft_consensus.cc:2229] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.653506 28654 raft_consensus.cc:2258] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.653798 28654 ts_tablet_manager.cc:1875] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.654162 28653 tablet_replica.cc:330] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.654225 28653 raft_consensus.cc:2229] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.654255 28653 raft_consensus.cc:2258] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.654798 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet c270bc02d9b04ddab310b305428fd48e
I20240503 02:09:18.654795 28661 ts_tablet_manager.cc:1888] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.654841 28661 log.cc:1196] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/8b7a955c305c4ac3a6ecd9248943a618
I20240503 02:09:18.654987 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet c270bc02d9b04ddab310b305428fd48e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.655097 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet c270bc02d9b04ddab310b305428fd48e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.655423 28653 ts_tablet_manager.cc:1875] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.655886 28654 ts_tablet_manager.cc:1888] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.655977 28654 log.cc:1196] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/609e5f3b76ac419ea61ae8a2ac42d22f
I20240503 02:09:18.656203 28654 ts_tablet_manager.cc:1909] T 609e5f3b76ac419ea61ae8a2ac42d22f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.656570 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 609e5f3b76ac419ea61ae8a2ac42d22f (table default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]) successfully deleted
I20240503 02:09:18.656600 28653 ts_tablet_manager.cc:1888] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.656749 28654 tablet_replica.cc:330] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.656831 28655 tablet_bootstrap.cc:492] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.656841 28654 raft_consensus.cc:2229] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.656883 28655 ts_tablet_manager.cc:1367] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.000s
I20240503 02:09:18.657035 28655 raft_consensus.cc:348] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.657110 28655 raft_consensus.cc:374] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.657138 28655 raft_consensus.cc:729] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.657173 28655 consensus_queue.cc:260] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.657261 28655 ts_tablet_manager.cc:1398] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:18.657347 28655 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.657059 28654 raft_consensus.cc:2258] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.657809 28655 tablet_bootstrap.cc:654] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.657969 28654 ts_tablet_manager.cc:1875] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.655383 28661 ts_tablet_manager.cc:1909] T 8b7a955c305c4ac3a6ecd9248943a618 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.658625 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 8b7a955c305c4ac3a6ecd9248943a618 (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:18.658779 28661 tablet_replica.cc:330] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.658859 28661 raft_consensus.cc:2229] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.658934 28661 pending_rounds.cc:62] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Trying to abort 1 pending ops.
I20240503 02:09:18.658982 28661 pending_rounds.cc:68] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420041210675200 op_type: NO_OP noop_request { }
I20240503 02:09:18.659060 28661 raft_consensus.cc:2866] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.659125 28661 raft_consensus.cc:2258] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.659549 28661 ts_tablet_manager.cc:1875] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.659579 28654 ts_tablet_manager.cc:1888] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.659679 28654 log.cc:1196] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/8b7a955c305c4ac3a6ecd9248943a618
I20240503 02:09:18.659845 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e37c4ea5fa1f4c8aa349246c1e52ec38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.659960 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e37c4ea5fa1f4c8aa349246c1e52ec38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.660018 28654 ts_tablet_manager.cc:1909] T 8b7a955c305c4ac3a6ecd9248943a618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.660357 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 8b7a955c305c4ac3a6ecd9248943a618 (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:18.660441 28654 tablet_replica.cc:330] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.660542 28654 raft_consensus.cc:2229] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.660632 28654 raft_consensus.cc:2258] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.660867 28654 ts_tablet_manager.cc:1875] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.661674 28661 ts_tablet_manager.cc:1888] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.661942 28661 log.cc:1196] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/c270bc02d9b04ddab310b305428fd48e
I20240503 02:09:18.661962 27427 tablet_service.cc:1514] Processing DeleteTablet for tablet c270bc02d9b04ddab310b305428fd48e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.662191 28661 ts_tablet_manager.cc:1909] T c270bc02d9b04ddab310b305428fd48e P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.662493 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet c270bc02d9b04ddab310b305428fd48e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.662619 28654 ts_tablet_manager.cc:1888] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.662714 28654 log.cc:1196] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/f56dc81ff4644d18a0105722f2f2f1e7
I20240503 02:09:18.663034 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2 (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.663126 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.663363 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet f56dc81ff4644d18a0105722f2f2f1e7 (table default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]) successfully deleted
I20240503 02:09:18.663550 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2 (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.663641 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.663919 28653 log.cc:1196] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/8b7a955c305c4ac3a6ecd9248943a618
I20240503 02:09:18.664093 28653 ts_tablet_manager.cc:1909] T 8b7a955c305c4ac3a6ecd9248943a618 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.664299 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 8b7a955c305c4ac3a6ecd9248943a618 (table default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]) successfully deleted
I20240503 02:09:18.664819 28664 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.664865 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 32314d265c0341d98f710e29f7786fbf (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.664944 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32314d265c0341d98f710e29f7786fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.665167 28664 tablet_bootstrap.cc:654] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.665477 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 56fd56a81cf4467c937e670ee2703c47 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.665552 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fd56a81cf4467c937e670ee2703c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.665700 28664 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.665740 28664 ts_tablet_manager.cc:1367] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.665874 28664 raft_consensus.cc:348] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.665931 28664 raft_consensus.cc:374] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.665948 28664 raft_consensus.cc:729] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.665985 28664 consensus_queue.cc:260] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.666067 28664 ts_tablet_manager.cc:1398] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.666072 27554 tablet_service.cc:1467] Processing CreateTablet for tablet 0b786eb5fd924ef1a7ae99a7761239c9 (DEFAULT_TABLE table=default.Table_e37c4ea5fa1f4c8aa349246c1e52ec38 [id=413b0771e62941759c7c3131e0629f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.666129 28664 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.666164 27554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b786eb5fd924ef1a7ae99a7761239c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.666476 28664 tablet_bootstrap.cc:654] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.667021 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 0b786eb5fd924ef1a7ae99a7761239c9 (DEFAULT_TABLE table=default.Table_e37c4ea5fa1f4c8aa349246c1e52ec38 [id=413b0771e62941759c7c3131e0629f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.667111 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b786eb5fd924ef1a7ae99a7761239c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.667362 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 56fd56a81cf4467c937e670ee2703c47 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.667441 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fd56a81cf4467c937e670ee2703c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.668236 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2 (DEFAULT_TABLE table=default.Table_11456da080574c88b77b8871e40fcfdf [id=8f0ad45fe2434d6982a2dd2aeda40680]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.668452 27198 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2d4d8ce484e048a1b6b35bdbfac1a691" } modify_external_catalogs: true
I20240503 02:09:18.668531 27198 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_2d4d8ce484e048a1b6b35bdbfac1a691" } modify_external_catalogs: true
I20240503 02:09:18.668632 28656 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.666075 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 32314d265c0341d98f710e29f7786fbf (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.669121 28654 ts_tablet_manager.cc:1909] T f56dc81ff4644d18a0105722f2f2f1e7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.669621 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet f56dc81ff4644d18a0105722f2f2f1e7 (table default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]) successfully deleted
I20240503 02:09:18.669617 27198 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b0e7025bcc084d6d9a6d34e61b75d9c7
I20240503 02:09:18.669764 28654 tablet_replica.cc:330] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.669890 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.669988 28661 tablet_replica.cc:330] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.670150 28661 raft_consensus.cc:2229] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.670215 28661 raft_consensus.cc:2258] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.670231 27427 tablet_service.cc:1467] Processing CreateTablet for tablet 0b786eb5fd924ef1a7ae99a7761239c9 (DEFAULT_TABLE table=default.Table_e37c4ea5fa1f4c8aa349246c1e52ec38 [id=413b0771e62941759c7c3131e0629f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.670316 27427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b786eb5fd924ef1a7ae99a7761239c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.670356 27422 tablet_service.cc:1467] Processing CreateTablet for tablet 56fd56a81cf4467c937e670ee2703c47 (DEFAULT_TABLE table=default.Table_ca2d781a10ed43a999c3abfa17fc4f91 [id=bdf8e7accaf643f790d3e502d68357bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.670431 27422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56fd56a81cf4467c937e670ee2703c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.670449 28653 tablet_replica.cc:330] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.670567 28653 raft_consensus.cc:2229] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.670692 28653 raft_consensus.cc:2258] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.670900 28653 ts_tablet_manager.cc:1875] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.671165 28655 tablet_bootstrap.cc:492] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.671200 28655 ts_tablet_manager.cc:1367] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240503 02:09:18.671280 28655 raft_consensus.cc:348] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.671329 28655 raft_consensus.cc:374] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.671345 28655 raft_consensus.cc:729] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.671386 28655 consensus_queue.cc:260] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.671458 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 32314d265c0341d98f710e29f7786fbf (DEFAULT_TABLE table=default.Table_0dad048e714d40d3ac909246ebf3c49f [id=3d57b52151c043018365befb16ab8c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.671592 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32314d265c0341d98f710e29f7786fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.672250 28653 ts_tablet_manager.cc:1888] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.672304 28653 log.cc:1196] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/c270bc02d9b04ddab310b305428fd48e
I20240503 02:09:18.672457 28653 ts_tablet_manager.cc:1909] T c270bc02d9b04ddab310b305428fd48e P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.672789 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.673251 28774 consensus_queue.cc:1035] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.673511 28653 tablet_replica.cc:330] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.673566 28653 raft_consensus.cc:2229] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.673607 28653 pending_rounds.cc:62] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Trying to abort 1 pending ops.
I20240503 02:09:18.673624 28653 pending_rounds.cc:68] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420040046919680 op_type: NO_OP noop_request { }
I20240503 02:09:18.673655 28653 raft_consensus.cc:2866] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.673672 28653 raft_consensus.cc:2258] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.673743 28661 ts_tablet_manager.cc:1875] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.674000 28653 ts_tablet_manager.cc:1875] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.671473 28655 ts_tablet_manager.cc:1398] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.674237 28655 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.674234 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet c270bc02d9b04ddab310b305428fd48e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.674479 28655 tablet_bootstrap.cc:654] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.674845 28774 consensus_queue.cc:1035] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.674880 28661 ts_tablet_manager.cc:1888] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.674924 28661 log.cc:1196] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b0e7025bcc084d6d9a6d34e61b75d9c7
I20240503 02:09:18.675061 28661 ts_tablet_manager.cc:1909] T b0e7025bcc084d6d9a6d34e61b75d9c7 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.675225 28654 raft_consensus.cc:2229] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.675487 28654 raft_consensus.cc:2258] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.675479 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.675606 28774 raft_consensus.cc:482] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.675649 28774 raft_consensus.cc:504] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.675753 28774 leader_election.cc:288] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.675884 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e040f4320b542cf9c6346447e1a66a4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.675942 27447 raft_consensus.cc:2454] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.676066 27659 leader_election.cc:302] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.676187 28693 raft_consensus.cc:2781] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.676209 28693 raft_consensus.cc:482] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.676225 28693 raft_consensus.cc:3037] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.676558 28410 raft_consensus.cc:482] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.676601 28410 raft_consensus.cc:504] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.676697 28410 leader_election.cc:288] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.676796 28693 raft_consensus.cc:504] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.676882 28693 leader_election.cc:288] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.676941 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88e8d58093e42dda61d4a08329c5ee9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.676990 27447 raft_consensus.cc:2454] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.677109 27524 leader_election.cc:302] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.677173 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_dc6bd0e5aca24f32870b87377275df2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.677291 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc6bd0e5aca24f32870b87377275df2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.677304 28653 ts_tablet_manager.cc:1888] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.677403 28653 log.cc:1196] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b0e7025bcc084d6d9a6d34e61b75d9c7
I20240503 02:09:18.677512 29241 raft_consensus.cc:2781] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.677544 29241 raft_consensus.cc:482] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.677563 29241 raft_consensus.cc:3037] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.677599 28653 ts_tablet_manager.cc:1909] T b0e7025bcc084d6d9a6d34e61b75d9c7 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.678046 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e040f4320b542cf9c6346447e1a66a4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.678117 27582 raft_consensus.cc:2454] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.678053 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e040f4320b542cf9c6346447e1a66a4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.678172 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e040f4320b542cf9c6346447e1a66a4" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.678215 27450 raft_consensus.cc:3037] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.678225 27587 raft_consensus.cc:3037] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.678129 28654 ts_tablet_manager.cc:1875] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.678755 27450 raft_consensus.cc:2454] T 3e040f4320b542cf9c6346447e1a66a4 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.678755 27587 raft_consensus.cc:2454] T 3e040f4320b542cf9c6346447e1a66a4 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.678941 27659 leader_election.cc:302] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.679311 28774 raft_consensus.cc:2781] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.679361 28774 raft_consensus.cc:686] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.679406 28774 consensus_queue.cc:237] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.681480 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88e8d58093e42dda61d4a08329c5ee9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.681545 27722 raft_consensus.cc:2454] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.681520 28410 consensus_queue.cc:1035] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.681677 27197 catalog_manager.cc:5484] T 3e040f4320b542cf9c6346447e1a66a4 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.678140 29241 raft_consensus.cc:504] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.682240 29241 leader_election.cc:288] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.682551 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88e8d58093e42dda61d4a08329c5ee9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.682622 27450 raft_consensus.cc:3037] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.683212 27450 raft_consensus.cc:2454] T b88e8d58093e42dda61d4a08329c5ee9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.683331 27524 leader_election.cc:302] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.683624 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88e8d58093e42dda61d4a08329c5ee9" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.683681 27720 raft_consensus.cc:3037] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.684293 27720 raft_consensus.cc:2454] T b88e8d58093e42dda61d4a08329c5ee9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.684525 29241 raft_consensus.cc:2781] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.684568 29241 raft_consensus.cc:686] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.684603 29241 consensus_queue.cc:237] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.685992 28655 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.686045 28655 ts_tablet_manager.cc:1367] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:18.686167 28655 raft_consensus.cc:348] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.686218 28655 raft_consensus.cc:374] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.686233 28655 raft_consensus.cc:729] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.686266 28655 consensus_queue.cc:260] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.686343 28655 ts_tablet_manager.cc:1398] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.686396 28655 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.686955 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a69cf47d9a00412f9dbfcf0190253e49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.687078 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a69cf47d9a00412f9dbfcf0190253e49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.668359 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a7abb0f315e47cdaff61e8e5b22d1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.689440 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32314d265c0341d98f710e29f7786fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.689647 28774 consensus_queue.cc:1035] T 0d5f5d7b273241e097bb196744e6f566 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.670184 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet b0e7025bcc084d6d9a6d34e61b75d9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.690650 28655 tablet_bootstrap.cc:654] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.691012 28656 tablet_bootstrap.cc:654] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.691313 27703 tablet_service.cc:1467] Processing CreateTablet for tablet cf8fdb2245a44bffa644a65f11bd034e (DEFAULT_TABLE table=default.Table_dc6bd0e5aca24f32870b87377275df2e [id=bbf480fce48844cab6b10dc6bc430ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.691355 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.691421 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8fdb2245a44bffa644a65f11bd034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.691560 27425 tablet_service.cc:1467] Processing CreateTablet for tablet cf8fdb2245a44bffa644a65f11bd034e (DEFAULT_TABLE table=default.Table_dc6bd0e5aca24f32870b87377275df2e [id=bbf480fce48844cab6b10dc6bc430ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.691650 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8fdb2245a44bffa644a65f11bd034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.691896 28664 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.691936 28664 ts_tablet_manager.cc:1367] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.001s
I20240503 02:09:18.692071 28664 raft_consensus.cc:348] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.692116 28664 raft_consensus.cc:374] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.692131 28664 raft_consensus.cc:729] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.692168 28664 consensus_queue.cc:260] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.692258 28664 ts_tablet_manager.cc:1398] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.692305 28664 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.692764 28664 tablet_bootstrap.cc:654] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.692847 27557 tablet_service.cc:1467] Processing CreateTablet for tablet cf8fdb2245a44bffa644a65f11bd034e (DEFAULT_TABLE table=default.Table_dc6bd0e5aca24f32870b87377275df2e [id=bbf480fce48844cab6b10dc6bc430ddd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.692945 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8fdb2245a44bffa644a65f11bd034e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.694547 27201 catalog_manager.cc:5484] T b88e8d58093e42dda61d4a08329c5ee9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.694693 27425 tablet_service.cc:1467] Processing CreateTablet for tablet ecaef6b27dba481dbab9e88231f81c20 (DEFAULT_TABLE table=default.Table_a69cf47d9a00412f9dbfcf0190253e49 [id=c594eef6918c4cd5b0acc753502d0077]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.694773 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaef6b27dba481dbab9e88231f81c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.694836 27703 tablet_service.cc:1467] Processing CreateTablet for tablet ecaef6b27dba481dbab9e88231f81c20 (DEFAULT_TABLE table=default.Table_a69cf47d9a00412f9dbfcf0190253e49 [id=c594eef6918c4cd5b0acc753502d0077]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.694911 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaef6b27dba481dbab9e88231f81c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.695303 28655 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.695348 28655 ts_tablet_manager.cc:1367] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:18.695441 28655 raft_consensus.cc:348] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.695497 28655 raft_consensus.cc:374] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.695514 28655 raft_consensus.cc:729] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.695554 28655 consensus_queue.cc:260] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.695641 28655 ts_tablet_manager.cc:1398] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.695719 28655 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.695955 28655 tablet_bootstrap.cc:654] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.696156 27557 tablet_service.cc:1467] Processing CreateTablet for tablet ecaef6b27dba481dbab9e88231f81c20 (DEFAULT_TABLE table=default.Table_a69cf47d9a00412f9dbfcf0190253e49 [id=c594eef6918c4cd5b0acc753502d0077]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.696245 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaef6b27dba481dbab9e88231f81c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.697444 28774 consensus_queue.cc:1035] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.697595 27848 consensus_queue.cc:1035] T dcc8f22f822e4b3e829bba41bd702f0c P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.699553 28654 ts_tablet_manager.cc:1888] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.699604 28654 log.cc:1196] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/c270bc02d9b04ddab310b305428fd48e
I20240503 02:09:18.699753 28654 ts_tablet_manager.cc:1909] T c270bc02d9b04ddab310b305428fd48e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.700683 28654 tablet_replica.cc:330] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.700798 28654 raft_consensus.cc:2229] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.700855 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet c270bc02d9b04ddab310b305428fd48e (table default.Table_8812ecdff34e469998f5419dcb07669b [id=bec13ce7f26d46b8948197a2c7434e24]) successfully deleted
I20240503 02:09:18.701030 28664 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.701076 28664 ts_tablet_manager.cc:1367] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:18.701117 27853 consensus_queue.cc:1035] T 6f7ccfcbdc854a09b17f6b03897ec63f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.701210 28664 raft_consensus.cc:348] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.701269 28664 raft_consensus.cc:374] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.701287 28664 raft_consensus.cc:729] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.700876 28654 raft_consensus.cc:2258] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.701359 28664 consensus_queue.cc:260] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.701462 28664 ts_tablet_manager.cc:1398] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.701666 28654 ts_tablet_manager.cc:1875] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.702626 28664 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.702998 28664 tablet_bootstrap.cc:654] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.704018 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_c4bf80f9758a4ffe93ff7c1c408beb2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.704134 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4bf80f9758a4ffe93ff7c1c408beb2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.704131 28654 ts_tablet_manager.cc:1888] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.704176 28654 log.cc:1196] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b0e7025bcc084d6d9a6d34e61b75d9c7
I20240503 02:09:18.704406 28654 ts_tablet_manager.cc:1909] T b0e7025bcc084d6d9a6d34e61b75d9c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.704761 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b0e7025bcc084d6d9a6d34e61b75d9c7 (table default.Table_2d4d8ce484e048a1b6b35bdbfac1a691 [id=c6b540cc05554c2a98a41abaf0e46b0f]) successfully deleted
I20240503 02:09:18.706276 28656 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.706324 28656 ts_tablet_manager.cc:1367] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240503 02:09:18.706454 28656 raft_consensus.cc:348] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.706506 28656 raft_consensus.cc:374] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.706542 28656 raft_consensus.cc:729] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.706576 28656 consensus_queue.cc:260] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.706670 28656 ts_tablet_manager.cc:1398] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.706741 28656 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.706934 28766 raft_consensus.cc:482] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.706977 28766 raft_consensus.cc:504] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.707064 28766 leader_election.cc:288] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.707098 28656 tablet_bootstrap.cc:654] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.707355 28774 consensus_queue.cc:1035] T 59aa5f0d43d14cf8bdafc8bb7fedb0f7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.707741 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.707916 28767 raft_consensus.cc:482] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.707965 28767 raft_consensus.cc:504] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.708056 28767 leader_election.cc:288] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.708194 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9201a0e5862b41878733a6f690600c66" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.708262 27720 raft_consensus.cc:2379] T 9201a0e5862b41878733a6f690600c66 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in this term.
I20240503 02:09:18.708312 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e343d86baf43ee9264f136015eb7db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.708372 27722 raft_consensus.cc:2454] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.708384 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e343d86baf43ee9264f136015eb7db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.708437 27582 raft_consensus.cc:2454] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.708524 27386 leader_election.cc:302] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.708611 28767 raft_consensus.cc:2781] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.708650 28767 raft_consensus.cc:482] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.708667 28767 raft_consensus.cc:3037] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.709123 28655 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.709161 28655 ts_tablet_manager.cc:1367] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:18.709249 28655 raft_consensus.cc:348] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.709300 28655 raft_consensus.cc:374] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.709360 28655 raft_consensus.cc:729] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.709416 27388 leader_election.cc:302] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.709553 28409 raft_consensus.cc:3037] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.709396 28655 consensus_queue.cc:260] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.709797 28655 ts_tablet_manager.cc:1398] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.709851 28655 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.710089 28655 tablet_bootstrap.cc:654] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.710206 28409 raft_consensus.cc:2726] T 9201a0e5862b41878733a6f690600c66 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.709306 28767 raft_consensus.cc:504] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.710500 28767 leader_election.cc:288] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.710809 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e343d86baf43ee9264f136015eb7db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.710867 27582 raft_consensus.cc:3037] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.711517 27582 raft_consensus.cc:2454] T a6e343d86baf43ee9264f136015eb7db P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.712280 27386 leader_election.cc:302] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.712296 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e343d86baf43ee9264f136015eb7db" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.712344 27720 raft_consensus.cc:3037] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.712424 29550 raft_consensus.cc:2781] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.712457 29550 raft_consensus.cc:686] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.712488 29550 consensus_queue.cc:237] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.712772 27201 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e5186cceee884329aa1427b0bd76d338"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.712872 27201 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5186cceee884329aa1427b0bd76d338 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.713039 27720 raft_consensus.cc:2454] T a6e343d86baf43ee9264f136015eb7db P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.713960 28656 tablet_bootstrap.cc:492] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.714013 28656 ts_tablet_manager.cc:1367] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240503 02:09:18.714129 28656 raft_consensus.cc:348] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.714200 28656 raft_consensus.cc:374] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.714216 28656 raft_consensus.cc:729] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.714262 28656 consensus_queue.cc:260] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.714354 28656 ts_tablet_manager.cc:1398] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.714409 28656 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.714733 28656 tablet_bootstrap.cc:654] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.714824 27197 catalog_manager.cc:5484] T a6e343d86baf43ee9264f136015eb7db P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.715595 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5b46cbf4bc4b37b463616d0f8c5108 (DEFAULT_TABLE table=default.Table_c4bf80f9758a4ffe93ff7c1c408beb2f [id=4fd56130ae2143b188277bd4e418dc87]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.715690 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5b46cbf4bc4b37b463616d0f8c5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.715916 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9faf33c75bfc4be5ac4acd77e380e778"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.716012 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9faf33c75bfc4be5ac4acd77e380e778 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.716127 27201 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_16a9f2e70b254f8ebbe44ebdbacc3db2" } new_table_name: "default.Table_594917e4550c43fd98e1db65db890c71" modify_external_catalogs: true
I20240503 02:09:18.716485 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5b46cbf4bc4b37b463616d0f8c5108 (DEFAULT_TABLE table=default.Table_c4bf80f9758a4ffe93ff7c1c408beb2f [id=4fd56130ae2143b188277bd4e418dc87]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.716531 28655 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.716572 28655 ts_tablet_manager.cc:1367] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.716576 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5b46cbf4bc4b37b463616d0f8c5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.716674 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 5d5b46cbf4bc4b37b463616d0f8c5108 (DEFAULT_TABLE table=default.Table_c4bf80f9758a4ffe93ff7c1c408beb2f [id=4fd56130ae2143b188277bd4e418dc87]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.716672 28655 raft_consensus.cc:348] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.716724 28655 raft_consensus.cc:374] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.716743 28655 raft_consensus.cc:729] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.716755 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d5b46cbf4bc4b37b463616d0f8c5108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.716780 28655 consensus_queue.cc:260] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.716873 28655 ts_tablet_manager.cc:1398] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.716929 28655 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.717142 28664 tablet_bootstrap.cc:492] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.717180 28664 ts_tablet_manager.cc:1367] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240503 02:09:18.717209 28655 tablet_bootstrap.cc:654] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.717311 28664 raft_consensus.cc:348] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.717383 28664 raft_consensus.cc:374] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.717401 28664 raft_consensus.cc:729] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.717438 28664 consensus_queue.cc:260] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.717567 28664 ts_tablet_manager.cc:1398] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.717633 28664 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.718000 28664 tablet_bootstrap.cc:654] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.718364 28767 raft_consensus.cc:482] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.718408 28767 raft_consensus.cc:504] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.718483 28767 leader_election.cc:288] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.718997 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bec77949895463791227ccf6e309f27" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.719070 27585 raft_consensus.cc:2454] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.719218 27386 leader_election.cc:302] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.719352 28767 raft_consensus.cc:2781] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.719383 28767 raft_consensus.cc:482] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.719400 28767 raft_consensus.cc:3037] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.719974 28767 raft_consensus.cc:504] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.720072 28767 leader_election.cc:288] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.720201 28740 consensus_queue.cc:1035] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.720338 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bec77949895463791227ccf6e309f27" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.720384 27585 raft_consensus.cc:3037] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.721014 27585 raft_consensus.cc:2454] T 8bec77949895463791227ccf6e309f27 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.721141 27386 leader_election.cc:302] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.721220 28409 raft_consensus.cc:2781] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.721254 28409 raft_consensus.cc:686] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.721263 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bec77949895463791227ccf6e309f27" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.721287 28409 consensus_queue.cc:237] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.721312 27720 raft_consensus.cc:2454] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.721499 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a3528014d434455e9b57fd9e6cf2f761"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.721601 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3528014d434455e9b57fd9e6cf2f761 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.721293 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bec77949895463791227ccf6e309f27" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.721796 27722 raft_consensus.cc:3037] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.722441 27722 raft_consensus.cc:2454] T 8bec77949895463791227ccf6e309f27 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.722605 28656 tablet_bootstrap.cc:492] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.722657 28656 ts_tablet_manager.cc:1367] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20240503 02:09:18.722790 28656 raft_consensus.cc:348] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.722865 28656 raft_consensus.cc:374] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.722883 28656 raft_consensus.cc:729] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.722916 28656 consensus_queue.cc:260] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.723012 28656 ts_tablet_manager.cc:1398] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.723062 28656 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.723416 28656 tablet_bootstrap.cc:654] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.724812 29643 consensus_queue.cc:1035] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.725092 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ec9cffa2fba342539c76839bbc1383cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.725206 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec9cffa2fba342539c76839bbc1383cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.725631 28655 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.725670 28655 ts_tablet_manager.cc:1367] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240503 02:09:18.725804 28655 raft_consensus.cc:348] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.725876 28655 raft_consensus.cc:374] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.725898 28655 raft_consensus.cc:729] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.725929 28655 consensus_queue.cc:260] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.726011 28655 ts_tablet_manager.cc:1398] T cf8fdb2245a44bffa644a65f11bd034e P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.726068 28655 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.726416 28655 tablet_bootstrap.cc:654] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.728116 28664 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.728166 28664 ts_tablet_manager.cc:1367] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240503 02:09:18.728317 28664 raft_consensus.cc:348] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.728379 28664 raft_consensus.cc:374] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.728400 28664 raft_consensus.cc:729] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.728438 28664 consensus_queue.cc:260] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.728538 28664 ts_tablet_manager.cc:1398] T cf8fdb2245a44bffa644a65f11bd034e P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.728617 28664 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.728818 28656 tablet_bootstrap.cc:492] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.728870 28656 ts_tablet_manager.cc:1367] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20240503 02:09:18.728989 28656 raft_consensus.cc:348] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.729041 28664 tablet_bootstrap.cc:654] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.729048 28656 raft_consensus.cc:374] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.729068 28656 raft_consensus.cc:729] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.729103 28656 consensus_queue.cc:260] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.729207 28656 ts_tablet_manager.cc:1398] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.729264 28656 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.729630 28656 tablet_bootstrap.cc:654] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.729662 27425 tablet_service.cc:1467] Processing CreateTablet for tablet e790208954e6433490eff4bf38b49835 (DEFAULT_TABLE table=default.Table_e5186cceee884329aa1427b0bd76d338 [id=628aafa967634f76bf9e0a25b03050dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.729749 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790208954e6433490eff4bf38b49835. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.729921 30009 tablet.cc:1722] T 5e1800d45b4f4593aa21127c2fc93f56 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.730176 27557 tablet_service.cc:1467] Processing CreateTablet for tablet e790208954e6433490eff4bf38b49835 (DEFAULT_TABLE table=default.Table_e5186cceee884329aa1427b0bd76d338 [id=628aafa967634f76bf9e0a25b03050dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.730263 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790208954e6433490eff4bf38b49835. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.730713 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e2e3322e21f94717961d13fd7b1971ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.730810 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2e3322e21f94717961d13fd7b1971ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.731220 28656 tablet_bootstrap.cc:492] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.731264 28656 ts_tablet_manager.cc:1367] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:18.731295 28740 consensus_queue.cc:1035] T 15819b1d3eec4d6fb3d1ef884508370a P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.731381 28656 raft_consensus.cc:348] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.731431 28656 raft_consensus.cc:374] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.731448 28656 raft_consensus.cc:729] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.731472 28656 consensus_queue.cc:260] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.731485 27703 tablet_service.cc:1467] Processing CreateTablet for tablet e790208954e6433490eff4bf38b49835 (DEFAULT_TABLE table=default.Table_e5186cceee884329aa1427b0bd76d338 [id=628aafa967634f76bf9e0a25b03050dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.731554 28656 ts_tablet_manager.cc:1398] T cf8fdb2245a44bffa644a65f11bd034e P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.731587 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e790208954e6433490eff4bf38b49835. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.731600 28656 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.731998 28656 tablet_bootstrap.cc:654] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.732437 28710 tablet.cc:1722] T 5e1800d45b4f4593aa21127c2fc93f56 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.734308 28664 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.734364 28664 ts_tablet_manager.cc:1367] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240503 02:09:18.734366 27196 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_663f85401b7743828f1c24e63c7f40ac" } modify_external_catalogs: true
I20240503 02:09:18.734423 27196 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_663f85401b7743828f1c24e63c7f40ac" } modify_external_catalogs: true
I20240503 02:09:18.734479 28664 raft_consensus.cc:348] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.734542 28664 raft_consensus.cc:374] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.734560 28664 raft_consensus.cc:729] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.734602 28664 consensus_queue.cc:260] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.734687 28664 ts_tablet_manager.cc:1398] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.734755 28664 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.735128 28664 tablet_bootstrap.cc:654] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.735630 30370 tablet.cc:1722] T 5e1800d45b4f4593aa21127c2fc93f56 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.735903 27565 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9452139ce42b1a5dbedb19ba89046 (DEFAULT_TABLE table=default.Table_9faf33c75bfc4be5ac4acd77e380e778 [id=74a1849da2f74345a978352fb6cc47c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.736001 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9452139ce42b1a5dbedb19ba89046. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.736920 28664 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.736925 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3cbd9d3dfc744f60a112172d69ea0cc0" } new_table_name: "default.Table_7e0b689ab5cc43eeac09d86b50a9c7db" modify_external_catalogs: true
I20240503 02:09:18.736963 28664 ts_tablet_manager.cc:1367] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.737095 28664 raft_consensus.cc:348] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.737147 28664 raft_consensus.cc:374] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.737164 28664 raft_consensus.cc:729] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.737197 28664 consensus_queue.cc:260] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.737258 27702 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9452139ce42b1a5dbedb19ba89046 (DEFAULT_TABLE table=default.Table_9faf33c75bfc4be5ac4acd77e380e778 [id=74a1849da2f74345a978352fb6cc47c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.737620 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc9cb8a18f4c299698671b49ec33bf (DEFAULT_TABLE table=default.Table_a3528014d434455e9b57fd9e6cf2f761 [id=0a78bf10c6f24929a3816cc208e3461a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.737708 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc9cb8a18f4c299698671b49ec33bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.737815 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc9cb8a18f4c299698671b49ec33bf (DEFAULT_TABLE table=default.Table_a3528014d434455e9b57fd9e6cf2f761 [id=0a78bf10c6f24929a3816cc208e3461a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.737308 28664 ts_tablet_manager.cc:1398] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.737887 28664 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.737890 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc9cb8a18f4c299698671b49ec33bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.737372 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9452139ce42b1a5dbedb19ba89046. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.738551 27196 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 0a549a5309d0451a8b58607ecdc3378a
I20240503 02:09:18.738749 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9452139ce42b1a5dbedb19ba89046 (DEFAULT_TABLE table=default.Table_9faf33c75bfc4be5ac4acd77e380e778 [id=74a1849da2f74345a978352fb6cc47c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.738840 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9452139ce42b1a5dbedb19ba89046. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.739231 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a549a5309d0451a8b58607ecdc3378a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.739687 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a549a5309d0451a8b58607ecdc3378a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.739750 28664 tablet_bootstrap.cc:654] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.739790 28653 tablet_replica.cc:330] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.739867 28653 raft_consensus.cc:2229] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.739899 28653 raft_consensus.cc:2258] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.739943 28661 tablet_replica.cc:330] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.740003 28661 raft_consensus.cc:2229] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.740015 28653 ts_tablet_manager.cc:1875] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.740037 28661 raft_consensus.cc:2258] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.740129 28661 ts_tablet_manager.cc:1875] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.740234 28664 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.740267 28664 ts_tablet_manager.cc:1367] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.740404 28664 raft_consensus.cc:348] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.740471 28664 raft_consensus.cc:374] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.740489 28664 raft_consensus.cc:729] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.740525 28664 consensus_queue.cc:260] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.740622 28664 ts_tablet_manager.cc:1398] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.740670 28664 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.740988 28664 tablet_bootstrap.cc:654] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.741215 28661 ts_tablet_manager.cc:1888] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.741259 28661 log.cc:1196] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0a549a5309d0451a8b58607ecdc3378a
I20240503 02:09:18.741448 28653 ts_tablet_manager.cc:1888] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.741492 28653 log.cc:1196] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0a549a5309d0451a8b58607ecdc3378a
I20240503 02:09:18.741503 28661 ts_tablet_manager.cc:1909] T 0a549a5309d0451a8b58607ecdc3378a P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.741513 28664 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.741569 28664 ts_tablet_manager.cc:1367] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.741691 28653 ts_tablet_manager.cc:1909] T 0a549a5309d0451a8b58607ecdc3378a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.741746 28664 raft_consensus.cc:348] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.741796 28664 raft_consensus.cc:374] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.741811 28664 raft_consensus.cc:729] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.741844 28664 consensus_queue.cc:260] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.741879 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0a549a5309d0451a8b58607ecdc3378a (table default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]) successfully deleted
I20240503 02:09:18.741935 28664 ts_tablet_manager.cc:1398] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.742933 27565 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94 (DEFAULT_TABLE table=default.Table_ec9cffa2fba342539c76839bbc1383cc [id=24dec5ffcf9b4748b359f34645d80c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.742980 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4483ac44174b05872e141c17b705da (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.743023 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.743090 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4483ac44174b05872e141c17b705da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.743472 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 58bc9cb8a18f4c299698671b49ec33bf (DEFAULT_TABLE table=default.Table_a3528014d434455e9b57fd9e6cf2f761 [id=0a78bf10c6f24929a3816cc208e3461a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.743553 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58bc9cb8a18f4c299698671b49ec33bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.744421 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4483ac44174b05872e141c17b705da (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.744508 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4483ac44174b05872e141c17b705da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.744578 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a549a5309d0451a8b58607ecdc3378a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.744688 28654 tablet_replica.cc:330] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.744743 28654 raft_consensus.cc:2229] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.744807 28654 raft_consensus.cc:2258] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.744874 28656 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.744902 28656 ts_tablet_manager.cc:1367] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240503 02:09:18.744921 28654 ts_tablet_manager.cc:1875] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.745028 28656 raft_consensus.cc:348] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.745074 28656 raft_consensus.cc:374] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.745088 28656 raft_consensus.cc:729] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.745121 28656 consensus_queue.cc:260] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.745198 28656 ts_tablet_manager.cc:1398] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.745245 28656 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.745612 28656 tablet_bootstrap.cc:654] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.745728 27197 catalog_manager.cc:5484] T 8bec77949895463791227ccf6e309f27 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.746063 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94 (DEFAULT_TABLE table=default.Table_ec9cffa2fba342539c76839bbc1383cc [id=24dec5ffcf9b4748b359f34645d80c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.746219 29558 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20240503 02:09:18.746264 28654 ts_tablet_manager.cc:1888] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.746304 28654 log.cc:1196] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0a549a5309d0451a8b58607ecdc3378a
I20240503 02:09:18.746492 28654 ts_tablet_manager.cc:1909] T 0a549a5309d0451a8b58607ecdc3378a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.746698 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94 (DEFAULT_TABLE table=default.Table_ec9cffa2fba342539c76839bbc1383cc [id=24dec5ffcf9b4748b359f34645d80c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.746778 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.746865 28767 raft_consensus.cc:482] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.746910 28767 raft_consensus.cc:504] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.746977 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4483ac44174b05872e141c17b705da (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.747005 28767 leader_election.cc:288] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.747052 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4483ac44174b05872e141c17b705da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.747469 28664 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.747741 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0a549a5309d0451a8b58607ecdc3378a (table default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]) successfully deleted
I20240503 02:09:18.747857 28664 tablet_bootstrap.cc:654] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.747925 28655 tablet_bootstrap.cc:492] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.747962 28655 ts_tablet_manager.cc:1367] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:18.748113 28655 raft_consensus.cc:348] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.748164 28655 raft_consensus.cc:374] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.748183 28655 raft_consensus.cc:729] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.748208 28655 consensus_queue.cc:260] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.748292 28655 ts_tablet_manager.cc:1398] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.748317 28710 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20240503 02:09:18.748370 28655 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.748407 30009 tablet.cc:1722] T f4a1100e58af4ce980576a3e78b98600 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20240503 02:09:18.748688 28655 tablet_bootstrap.cc:654] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.749462 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d5a0b2ecb96459aaf26b8cd138f2b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.750017 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.750078 27582 raft_consensus.cc:2379] T 4831e4a404b34b88b0511d2e5eca0897 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd4b79bd8b57443faa6fa5e464570188 in current term 1: Already voted for candidate c2be543b8aad4e80a58d6b4ab669faae in this term.
I20240503 02:09:18.750841 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4831e4a404b34b88b0511d2e5eca0897" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.750833 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_b8ea2f917e774cb3a0ad56468324a4f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.750932 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8ea2f917e774cb3a0ad56468324a4f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.751202 27388 leader_election.cc:302] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188; no voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.751215 28740 raft_consensus.cc:482] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.751252 28740 raft_consensus.cc:504] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.751297 28767 raft_consensus.cc:3037] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.751346 28740 leader_election.cc:288] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.751587 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32314d265c0341d98f710e29f7786fbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.751644 27582 raft_consensus.cc:2454] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.751924 28767 raft_consensus.cc:2726] T 4831e4a404b34b88b0511d2e5eca0897 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240503 02:09:18.751969 27656 leader_election.cc:302] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.752043 28740 raft_consensus.cc:2781] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.752066 28740 raft_consensus.cc:482] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.752081 28740 raft_consensus.cc:3037] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.752707 28740 raft_consensus.cc:504] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.752826 28740 leader_election.cc:288] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.753149 28656 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.753144 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32314d265c0341d98f710e29f7786fbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.753190 28656 ts_tablet_manager.cc:1367] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240503 02:09:18.753201 27452 raft_consensus.cc:3037] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.753315 28656 raft_consensus.cc:348] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.753398 28656 raft_consensus.cc:374] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.753414 28656 raft_consensus.cc:729] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.753444 28656 consensus_queue.cc:260] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.753523 28656 ts_tablet_manager.cc:1398] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.753573 28656 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.753826 27452 raft_consensus.cc:2454] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.753881 28656 tablet_bootstrap.cc:654] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.753960 27659 leader_election.cc:302] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.754912 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_594917e4550c43fd98e1db65db890c71 [id=24f3eb6c203b452cac076beda486824a] alter complete (version 1)
I20240503 02:09:18.755028 28656 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.755074 28656 ts_tablet_manager.cc:1367] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:18.755195 28656 raft_consensus.cc:348] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.755254 28656 raft_consensus.cc:374] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.755266 28656 raft_consensus.cc:729] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.755286 28656 consensus_queue.cc:260] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.755357 28656 ts_tablet_manager.cc:1398] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.755407 28656 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.755745 28656 tablet_bootstrap.cc:654] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.756541 28656 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.756582 28656 ts_tablet_manager.cc:1367] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.756688 28656 raft_consensus.cc:348] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.756732 28656 raft_consensus.cc:374] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.756749 28656 raft_consensus.cc:729] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.756788 28656 consensus_queue.cc:260] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.756872 28656 ts_tablet_manager.cc:1398] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.756913 28656 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.757261 28656 tablet_bootstrap.cc:654] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.757616 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32314d265c0341d98f710e29f7786fbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.757700 27447 raft_consensus.cc:2362] T 32314d265c0341d98f710e29f7786fbf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:18.758052 29550 consensus_queue.cc:1035] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.758131 28656 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.758139 29550 consensus_queue.cc:1035] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.758164 28656 ts_tablet_manager.cc:1367] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.758267 28656 raft_consensus.cc:348] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.758311 28656 raft_consensus.cc:374] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.758327 28656 raft_consensus.cc:729] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.758349 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0a549a5309d0451a8b58607ecdc3378a (table default.Table_663f85401b7743828f1c24e63c7f40ac [id=ce855f2cd9c4425d8fc0ba16a8e1dee5]) successfully deleted
I20240503 02:09:18.758360 28656 consensus_queue.cc:260] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.758446 28656 ts_tablet_manager.cc:1398] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.758491 28656 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.758828 28656 tablet_bootstrap.cc:654] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.758999 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_a3528014d434455e9b57fd9e6cf2f761" } new_table_name: "default.Table_6c53d6cacadf4be1b38892cb44c74321" modify_external_catalogs: true
I20240503 02:09:18.759290 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32314d265c0341d98f710e29f7786fbf" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.759353 27587 raft_consensus.cc:3037] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.759748 28656 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.759784 28656 ts_tablet_manager.cc:1367] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.759891 28656 raft_consensus.cc:348] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.759940 28656 raft_consensus.cc:374] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.759956 28656 raft_consensus.cc:729] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.759977 27587 raft_consensus.cc:2454] T 32314d265c0341d98f710e29f7786fbf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.759989 28656 consensus_queue.cc:260] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.760093 28656 ts_tablet_manager.cc:1398] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.760136 28656 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.760210 28740 raft_consensus.cc:2781] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.760252 28740 raft_consensus.cc:686] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.760285 28740 consensus_queue.cc:237] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.760483 28656 tablet_bootstrap.cc:654] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.760481 28767 raft_consensus.cc:482] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.760535 28767 raft_consensus.cc:504] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.760622 28767 leader_election.cc:288] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.760921 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_7e0b689ab5cc43eeac09d86b50a9c7db [id=5ff15871699b4869b63a0187224a6cd7] alter complete (version 5)
I20240503 02:09:18.760906 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dac6b3737de4d9e8b5063d82b6a4648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.760965 27720 raft_consensus.cc:2454] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
W20240503 02:09:18.761142 27198 catalog_manager.cc:5744] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58bc9cb8a18f4c299698671b49ec33bf consensus state has no leader
I20240503 02:09:18.761265 27388 leader_election.cc:302] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.761410 28693 raft_consensus.cc:482] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.761459 28693 raft_consensus.cc:504] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.761547 28693 leader_election.cc:288] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.761736 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fb873c91f543cf9297c29528eb5ed1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.761793 27447 raft_consensus.cc:2454] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.761996 27659 leader_election.cc:302] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.762166 28767 raft_consensus.cc:2781] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.762199 28767 raft_consensus.cc:482] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.762215 28767 raft_consensus.cc:3037] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.762354 28693 raft_consensus.cc:2781] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.762383 28693 raft_consensus.cc:482] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.762398 28693 raft_consensus.cc:3037] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.762389 28656 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.762426 28656 ts_tablet_manager.cc:1367] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240503 02:09:18.762533 28656 raft_consensus.cc:348] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.762567 28656 raft_consensus.cc:374] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.762578 28656 raft_consensus.cc:729] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.762609 28656 consensus_queue.cc:260] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.762704 28656 ts_tablet_manager.cc:1398] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.762773 28767 raft_consensus.cc:504] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.762885 28767 leader_election.cc:288] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.762972 28693 raft_consensus.cc:504] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.763075 28693 leader_election.cc:288] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.763054 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dac6b3737de4d9e8b5063d82b6a4648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.763110 27720 raft_consensus.cc:3037] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.763612 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fb873c91f543cf9297c29528eb5ed1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.763677 27447 raft_consensus.cc:3037] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.763775 27720 raft_consensus.cc:2454] T 3dac6b3737de4d9e8b5063d82b6a4648 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.763936 27588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dac6b3737de4d9e8b5063d82b6a4648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.763994 27588 raft_consensus.cc:3037] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.763967 27584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fb873c91f543cf9297c29528eb5ed1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.763942 27583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dac6b3737de4d9e8b5063d82b6a4648" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.764025 27584 raft_consensus.cc:3037] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.764290 27447 raft_consensus.cc:2454] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.764608 27588 raft_consensus.cc:2454] T 3dac6b3737de4d9e8b5063d82b6a4648 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.764699 27584 raft_consensus.cc:2454] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.764712 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17fb873c91f543cf9297c29528eb5ed1" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.764760 27587 raft_consensus.cc:2362] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1. Re-sending same reply.
I20240503 02:09:18.764864 27659 leader_election.cc:302] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.764928 28693 raft_consensus.cc:2781] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.764976 28693 raft_consensus.cc:686] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.765022 28693 consensus_queue.cc:237] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.765594 27388 leader_election.cc:302] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.766223 28767 raft_consensus.cc:2781] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.766278 28767 raft_consensus.cc:686] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.766314 28767 consensus_queue.cc:237] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.766323 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 51875ccc8e4e4ff98b74eaf7118c20c7 (DEFAULT_TABLE table=default.Table_b8ea2f917e774cb3a0ad56468324a4f1 [id=6aeeef6c68534261b03bd5ff5f457723]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.766407 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51875ccc8e4e4ff98b74eaf7118c20c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.767258 27197 catalog_manager.cc:5484] T 3dac6b3737de4d9e8b5063d82b6a4648 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.767777 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 51875ccc8e4e4ff98b74eaf7118c20c7 (DEFAULT_TABLE table=default.Table_b8ea2f917e774cb3a0ad56468324a4f1 [id=6aeeef6c68534261b03bd5ff5f457723]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.767871 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51875ccc8e4e4ff98b74eaf7118c20c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.768632 27195 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c7ed276358c14b26bfd22e4bf3f709e1" } new_table_name: "default.Table_4e42a56935c043d6becc676460c1dbe9" modify_external_catalogs: true
I20240503 02:09:18.768832 28656 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.768990 28664 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.769029 28664 ts_tablet_manager.cc:1367] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20240503 02:09:18.769157 28664 raft_consensus.cc:348] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.769207 28664 raft_consensus.cc:374] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.769223 28664 raft_consensus.cc:729] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.769255 28656 tablet_bootstrap.cc:654] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.769258 28664 consensus_queue.cc:260] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.769292 27200 catalog_manager.cc:5484] T 32314d265c0341d98f710e29f7786fbf P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.769381 28664 ts_tablet_manager.cc:1398] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.769431 28664 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.769483 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.769767 28664 tablet_bootstrap.cc:654] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.770411 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 51875ccc8e4e4ff98b74eaf7118c20c7 (DEFAULT_TABLE table=default.Table_b8ea2f917e774cb3a0ad56468324a4f1 [id=6aeeef6c68534261b03bd5ff5f457723]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.770511 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51875ccc8e4e4ff98b74eaf7118c20c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.770558 28655 tablet_bootstrap.cc:492] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.770601 28655 ts_tablet_manager.cc:1367] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:18.770794 28655 raft_consensus.cc:348] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.770848 28655 raft_consensus.cc:374] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.770865 28655 raft_consensus.cc:729] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.770897 28655 consensus_queue.cc:260] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.770989 28655 ts_tablet_manager.cc:1398] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20240503 02:09:18.771052 28655 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.771391 28655 tablet_bootstrap.cc:654] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.772924 27720 raft_consensus.cc:1261] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.773204 27587 raft_consensus.cc:1261] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.773461 28767 consensus_queue.cc:1035] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.774298 29550 consensus_queue.cc:1035] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.774942 27200 catalog_manager.cc:5484] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.775012 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 9fd59c2a068642c6af4742c990176172 (table default.Table_4e42a56935c043d6becc676460c1dbe9 [id=146e32948828496a8e05763492a6b34d]). Expected version 1 got 0
I20240503 02:09:18.775049 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:18.775082 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.776616 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 9fd59c2a068642c6af4742c990176172 (table default.Table_4e42a56935c043d6becc676460c1dbe9 [id=146e32948828496a8e05763492a6b34d]). Expected version 1 got 0
I20240503 02:09:18.777789 28664 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.777827 28664 ts_tablet_manager.cc:1367] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:18.777957 28664 raft_consensus.cc:348] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.778013 28664 raft_consensus.cc:374] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.778030 28664 raft_consensus.cc:729] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.778056 28664 consensus_queue.cc:260] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.778137 28664 ts_tablet_manager.cc:1398] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.778182 28664 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.778208 30009 tablet.cc:1722] T 9fd59c2a068642c6af4742c990176172 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.778512 28710 tablet.cc:1722] T 9fd59c2a068642c6af4742c990176172 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.778527 28664 tablet_bootstrap.cc:654] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.778790 27200 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_c148f7646e0c499787482efa0d32cb21" } new_table_name: "default.Table_0ef4b32f5d2d4d79a4afdb8745ec1460" modify_external_catalogs: true
I20240503 02:09:18.778923 29643 consensus_queue.cc:1035] T e6bb29693c414ac98223c93f543ad14f P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.779858 28655 tablet_bootstrap.cc:492] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.779897 28655 ts_tablet_manager.cc:1367] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.000s
I20240503 02:09:18.780055 28655 raft_consensus.cc:348] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.780108 28655 raft_consensus.cc:374] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.780125 28655 raft_consensus.cc:729] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.780150 28655 consensus_queue.cc:260] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.780251 28655 ts_tablet_manager.cc:1398] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.780303 28655 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.780679 28655 tablet_bootstrap.cc:654] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.781880 27195 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_4e42a56935c043d6becc676460c1dbe9 [id=146e32948828496a8e05763492a6b34d] alter complete (version 1)
I20240503 02:09:18.782291 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 9fd59c2a068642c6af4742c990176172 (table default.Table_4e42a56935c043d6becc676460c1dbe9 [id=146e32948828496a8e05763492a6b34d]). Expected version 1 got 0
I20240503 02:09:18.784588 27848 raft_consensus.cc:482] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.784634 27848 raft_consensus.cc:504] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.784746 27848 leader_election.cc:288] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
W20240503 02:09:18.785009 27524 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
W20240503 02:09:18.785245 27523 leader_election.cc:341] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): Not found: Tablet not found: 203665395f3745f196fb393dd82b8fc9
I20240503 02:09:18.785286 27523 leader_election.cc:302] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db; no voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae
I20240503 02:09:18.785382 27848 raft_consensus.cc:2726] T 203665395f3745f196fb393dd82b8fc9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240503 02:09:18.786481 28664 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.786535 28664 ts_tablet_manager.cc:1367] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240503 02:09:18.786648 28664 raft_consensus.cc:348] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.786700 28664 raft_consensus.cc:374] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.786716 28664 raft_consensus.cc:729] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.786715 30009 tablet.cc:1722] T 9654b6fa66a84fffb440477a95a0f3a9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.786747 28664 consensus_queue.cc:260] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.786844 28664 ts_tablet_manager.cc:1398] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.786921 28664 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.787410 28664 tablet_bootstrap.cc:654] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.788422 29550 consensus_queue.cc:1035] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.789229 28710 tablet.cc:1722] T 9654b6fa66a84fffb440477a95a0f3a9 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.790398 29550 consensus_queue.cc:1035] T 6c869e9a2d9a4472a65d57af51b491a1 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.790733 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bd478bdb5a454d489940b1dddf851c66" } modify_external_catalogs: true
I20240503 02:09:18.790798 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_bd478bdb5a454d489940b1dddf851c66" } modify_external_catalogs: true
I20240503 02:09:18.790923 28767 consensus_queue.cc:1035] T e76ee4b127b944ee8611ac188dbcc087 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.792799 27198 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_0ef4b32f5d2d4d79a4afdb8745ec1460 [id=bf423777e8d14b688a677e37aab7a477] alter complete (version 1)
I20240503 02:09:18.793594 28655 tablet_bootstrap.cc:492] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.793640 28655 ts_tablet_manager.cc:1367] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.000s
I20240503 02:09:18.793759 28655 raft_consensus.cc:348] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.793807 28655 raft_consensus.cc:374] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.793823 28655 raft_consensus.cc:729] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.793854 28655 consensus_queue.cc:260] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.793951 28655 ts_tablet_manager.cc:1398] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.794628 27200 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_989c040b372244d2875f77eed4f845ac" } modify_external_catalogs: true
I20240503 02:09:18.794684 27200 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_989c040b372244d2875f77eed4f845ac" } modify_external_catalogs: true
I20240503 02:09:18.794698 28655 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.795029 28655 tablet_bootstrap.cc:654] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.795078 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b363bfa5d6f04d658cec669b62811c4f
I20240503 02:09:18.795286 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet b363bfa5d6f04d658cec669b62811c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.795356 28654 tablet_replica.cc:330] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.795413 28654 raft_consensus.cc:2229] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.795455 28654 pending_rounds.cc:62] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Trying to abort 1 pending ops.
I20240503 02:09:18.795466 28654 pending_rounds.cc:68] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7023420040165588992 op_type: NO_OP noop_request { }
I20240503 02:09:18.795490 28654 raft_consensus.cc:2866] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:09:18.795509 28654 raft_consensus.cc:2258] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.795570 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet b363bfa5d6f04d658cec669b62811c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.795614 28656 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.795639 28661 tablet_replica.cc:330] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.795653 28656 ts_tablet_manager.cc:1367] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20240503 02:09:18.795691 28661 raft_consensus.cc:2229] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.795723 28661 raft_consensus.cc:2258] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.795768 28656 raft_consensus.cc:348] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.795817 28661 ts_tablet_manager.cc:1875] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.795823 28656 raft_consensus.cc:374] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.795842 28656 raft_consensus.cc:729] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.795876 28656 consensus_queue.cc:260] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.795994 28656 ts_tablet_manager.cc:1398] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.796020 28654 ts_tablet_manager.cc:1875] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.796339 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
W20240503 02:09:18.796443 27187 catalog_manager.cc:4628] Async tablet task AlterTable RPC for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 58bc9cb8a18f4c299698671b49ec33bf consensus state has no leader
I20240503 02:09:18.797230 28661 ts_tablet_manager.cc:1888] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.797281 28661 log.cc:1196] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b363bfa5d6f04d658cec669b62811c4f
I20240503 02:09:18.797344 28654 ts_tablet_manager.cc:1888] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.797386 28654 log.cc:1196] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b363bfa5d6f04d658cec669b62811c4f
I20240503 02:09:18.797474 28661 ts_tablet_manager.cc:1909] T b363bfa5d6f04d658cec669b62811c4f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.797580 28654 ts_tablet_manager.cc:1909] T b363bfa5d6f04d658cec669b62811c4f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.798288 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ad1bbe9bd4a14f40ac340c5189f6056d
I20240503 02:09:18.798488 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b363bfa5d6f04d658cec669b62811c4f (table default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]) successfully deleted
I20240503 02:09:18.798650 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.798694 28664 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.798732 28664 ts_tablet_manager.cc:1367] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240503 02:09:18.798869 28664 raft_consensus.cc:348] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.798920 28664 raft_consensus.cc:374] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.798936 28664 raft_consensus.cc:729] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.798969 28664 consensus_queue.cc:260] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.799068 28664 ts_tablet_manager.cc:1398] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.799289 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.799492 30370 tablet.cc:1722] T 9654b6fa66a84fffb440477a95a0f3a9 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.799736 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet ad1bbe9bd4a14f40ac340c5189f6056d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.799825 28653 tablet_replica.cc:330] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.799896 28653 raft_consensus.cc:2229] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.799968 28653 raft_consensus.cc:2258] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.799813 27447 raft_consensus.cc:1261] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.800139 29643 consensus_queue.cc:1035] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.799844 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet b363bfa5d6f04d658cec669b62811c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.800487 28653 ts_tablet_manager.cc:1875] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.800647 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.800854 28661 tablet_replica.cc:330] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.800912 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b363bfa5d6f04d658cec669b62811c4f (table default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]) successfully deleted
I20240503 02:09:18.800922 28661 raft_consensus.cc:2229] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.800953 28661 raft_consensus.cc:2258] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.801183 28661 ts_tablet_manager.cc:1875] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.801779 28653 ts_tablet_manager.cc:1888] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.801820 28653 log.cc:1196] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ad1bbe9bd4a14f40ac340c5189f6056d
I20240503 02:09:18.801991 28653 ts_tablet_manager.cc:1909] T ad1bbe9bd4a14f40ac340c5189f6056d P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.802428 28661 ts_tablet_manager.cc:1888] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.802474 28661 log.cc:1196] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ad1bbe9bd4a14f40ac340c5189f6056d
I20240503 02:09:18.803035 30370 tablet.cc:1722] T 9fd59c2a068642c6af4742c990176172 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.803342 28661 ts_tablet_manager.cc:1909] T ad1bbe9bd4a14f40ac340c5189f6056d P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.803342 27200 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:18.803829 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_989c040b372244d2875f77eed4f845ac [id=15574575df68415693e80eaea9955e92]) successfully deleted
I20240503 02:09:18.804607 28653 tablet_replica.cc:330] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.804669 28653 raft_consensus.cc:2229] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.804695 28653 raft_consensus.cc:2258] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.804790 28653 ts_tablet_manager.cc:1875] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.805557 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_989c040b372244d2875f77eed4f845ac [id=15574575df68415693e80eaea9955e92]) successfully deleted
I20240503 02:09:18.805613 28654 tablet_replica.cc:330] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.805667 28654 raft_consensus.cc:2229] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.805701 28654 raft_consensus.cc:2258] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.805994 28653 ts_tablet_manager.cc:1888] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.806047 28653 log.cc:1196] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b363bfa5d6f04d658cec669b62811c4f
I20240503 02:09:18.806190 28653 ts_tablet_manager.cc:1909] T b363bfa5d6f04d658cec669b62811c4f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.806509 27582 raft_consensus.cc:1261] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer c2be543b8aad4e80a58d6b4ab669faae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.806751 28654 ts_tablet_manager.cc:1875] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.807987 28654 ts_tablet_manager.cc:1888] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240503 02:09:18.808041 28654 log.cc:1196] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ad1bbe9bd4a14f40ac340c5189f6056d
I20240503 02:09:18.808161 28767 consensus_queue.cc:1035] T 6540e09a284246ee92301479690ed27c P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.808233 28654 ts_tablet_manager.cc:1909] T ad1bbe9bd4a14f40ac340c5189f6056d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.808399 29643 consensus_queue.cc:1035] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.808554 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ad1bbe9bd4a14f40ac340c5189f6056d (table default.Table_989c040b372244d2875f77eed4f845ac [id=15574575df68415693e80eaea9955e92]) successfully deleted
I20240503 02:09:18.809615 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b363bfa5d6f04d658cec669b62811c4f (table default.Table_bd478bdb5a454d489940b1dddf851c66 [id=3885a2684cdd43dabf838635bc8dc65c]) successfully deleted
I20240503 02:09:18.809746 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e4483ac44174b05872e141c17b705da from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.810755 30370 tablet.cc:1722] T 17fb873c91f543cf9297c29528eb5ed1 P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.810874 28655 tablet_bootstrap.cc:492] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.810914 28655 ts_tablet_manager.cc:1367] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.000s
I20240503 02:09:18.811043 28655 raft_consensus.cc:348] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.811105 28655 raft_consensus.cc:374] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.811123 28655 raft_consensus.cc:729] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.811156 28655 consensus_queue.cc:260] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.811252 28655 ts_tablet_manager.cc:1398] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.811312 28655 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.811730 28693 raft_consensus.cc:482] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.811780 28655 tablet_bootstrap.cc:654] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.811777 28693 raft_consensus.cc:504] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.811868 28693 leader_election.cc:288] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.812053 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaef6b27dba481dbab9e88231f81c20" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.812134 27450 raft_consensus.cc:2454] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.812214 28710 tablet.cc:1722] T 17fb873c91f543cf9297c29528eb5ed1 P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.812685 27195 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 17fb873c91f543cf9297c29528eb5ed1 (table default.Table_981eecb56462490eaeee1cc6e0d2eaa5 [id=13def8c37808439d9ed2ae1c294b9da0]). Expected version 1 got 0
I20240503 02:09:18.813266 27659 leader_election.cc:302] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.813266 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 0e4483ac44174b05872e141c17b705da
I20240503 02:09:18.813364 28767 consensus_queue.cc:1035] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.813407 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 0e4483ac44174b05872e141c17b705da deleted and replaced by tablet e3db64d2695c4149887528b59b18df5f
I20240503 02:09:18.813619 27853 raft_consensus.cc:2781] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.813661 27853 raft_consensus.cc:482] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.813680 27853 raft_consensus.cc:3037] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.814302 27853 raft_consensus.cc:504] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.814397 27853 leader_election.cc:288] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.814962 27848 raft_consensus.cc:482] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.815004 27848 raft_consensus.cc:504] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.815104 27848 leader_election.cc:288] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.815263 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7abb0f315e47cdaff61e8e5b22d1a2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.815318 27722 raft_consensus.cc:2454] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.815353 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4483ac44174b05872e141c17b705da with delete_type TABLET_DATA_DELETED (replaced by tablet e3db64d2695c4149887528b59b18df5f) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.815434 28654 tablet_replica.cc:330] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.815459 27523 leader_election.cc:302] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.815485 28654 raft_consensus.cc:2229] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.815495 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaef6b27dba481dbab9e88231f81c20" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.815534 27848 raft_consensus.cc:2781] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.815546 27450 raft_consensus.cc:3037] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.815563 27848 raft_consensus.cc:482] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.815580 27848 raft_consensus.cc:3037] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.815608 28654 raft_consensus.cc:2258] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.815780 28654 ts_tablet_manager.cc:1875] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.816157 27450 raft_consensus.cc:2454] T ecaef6b27dba481dbab9e88231f81c20 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.816217 27848 raft_consensus.cc:504] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.816313 27848 leader_election.cc:288] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.816368 28655 tablet_bootstrap.cc:492] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.816411 28655 ts_tablet_manager.cc:1367] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.816538 28655 raft_consensus.cc:348] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.816592 28655 raft_consensus.cc:374] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.816608 28655 raft_consensus.cc:729] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.816640 28655 consensus_queue.cc:260] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.816716 28655 ts_tablet_manager.cc:1398] T 6d5a0b2ecb96459aaf26b8cd138f2b94 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.816761 28655 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.817018 28654 ts_tablet_manager.cc:1888] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.817067 28654 log.cc:1196] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/0e4483ac44174b05872e141c17b705da
I20240503 02:09:18.817093 28655 tablet_bootstrap.cc:654] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.817221 28654 ts_tablet_manager.cc:1909] T 0e4483ac44174b05872e141c17b705da P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.817355 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4483ac44174b05872e141c17b705da with delete_type TABLET_DATA_DELETED (replaced by tablet e3db64d2695c4149887528b59b18df5f) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.817525 27720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7abb0f315e47cdaff61e8e5b22d1a2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.817582 27720 raft_consensus.cc:3037] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.817692 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4483ac44174b05872e141c17b705da with delete_type TABLET_DATA_DELETED (replaced by tablet e3db64d2695c4149887528b59b18df5f) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.818190 27659 leader_election.cc:302] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.818238 27720 raft_consensus.cc:2454] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.818259 28693 raft_consensus.cc:2781] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.818297 28693 raft_consensus.cc:686] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.818332 28693 consensus_queue.cc:237] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.818392 27523 leader_election.cc:302] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.818450 28661 tablet_replica.cc:330] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.818455 27848 raft_consensus.cc:2781] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.818495 27848 raft_consensus.cc:686] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.818507 28661 raft_consensus.cc:2229] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.818533 28661 raft_consensus.cc:2258] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.818529 27848 consensus_queue.cc:237] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.818836 28774 raft_consensus.cc:482] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.818876 28774 raft_consensus.cc:504] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.818975 28774 leader_election.cc:288] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.819439 27692 tablet_service.cc:1467] Processing CreateTablet for tablet e3db64d2695c4149887528b59b18df5f (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.819532 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3db64d2695c4149887528b59b18df5f. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:18.819547 27190 catalog_manager.cc:4810] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): delete failed for tablet 0e4483ac44174b05872e141c17b705da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e4483ac44174b05872e141c17b705da already in progress: creating tablet
I20240503 02:09:18.819656 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51875ccc8e4e4ff98b74eaf7118c20c7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.819692 27586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaef6b27dba481dbab9e88231f81c20" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.819711 27587 raft_consensus.cc:2454] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.819732 27586 raft_consensus.cc:2454] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.819785 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaef6b27dba481dbab9e88231f81c20" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.819818 27585 raft_consensus.cc:3037] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.820298 28655 tablet_bootstrap.cc:492] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.820336 28655 ts_tablet_manager.cc:1367] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20240503 02:09:18.820446 27585 raft_consensus.cc:2454] T ecaef6b27dba481dbab9e88231f81c20 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.820479 28655 raft_consensus.cc:348] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.820537 28655 raft_consensus.cc:374] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.820554 28655 raft_consensus.cc:729] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.820585 28655 consensus_queue.cc:260] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.820668 28655 ts_tablet_manager.cc:1398] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.820745 28655 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.821084 28655 tablet_bootstrap.cc:654] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.821249 27656 leader_election.cc:302] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.821394 27853 raft_consensus.cc:2781] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.821439 27853 raft_consensus.cc:482] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.821456 27853 raft_consensus.cc:3037] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.821449 27557 tablet_service.cc:1467] Processing CreateTablet for tablet e3db64d2695c4149887528b59b18df5f (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.821491 28664 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.821528 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3db64d2695c4149887528b59b18df5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.821861 28664 tablet_bootstrap.cc:654] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.822033 27853 raft_consensus.cc:504] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.822141 27853 leader_election.cc:288] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.822489 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7abb0f315e47cdaff61e8e5b22d1a2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.822556 27447 raft_consensus.cc:3037] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.822662 28656 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.822690 30009 tablet.cc:1722] T 17fb873c91f543cf9297c29528eb5ed1 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.822876 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a7abb0f315e47cdaff61e8e5b22d1a2" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.822927 28656 tablet_bootstrap.cc:654] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.823158 27447 raft_consensus.cc:2454] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.823355 27425 tablet_service.cc:1467] Processing CreateTablet for tablet e3db64d2695c4149887528b59b18df5f (DEFAULT_TABLE table=default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.823446 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3db64d2695c4149887528b59b18df5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.824023 27200 catalog_manager.cc:5484] T 1a7abb0f315e47cdaff61e8e5b22d1a2 P 5cc801b4e0b047b3b4b30f8dd5c7c5db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2). New cstate: current_term: 1 leader_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.824790 27452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51875ccc8e4e4ff98b74eaf7118c20c7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.824854 27195 catalog_manager.cc:5484] T ecaef6b27dba481dbab9e88231f81c20 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.825603 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51875ccc8e4e4ff98b74eaf7118c20c7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.826280 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51875ccc8e4e4ff98b74eaf7118c20c7" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.826339 27585 raft_consensus.cc:3037] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.827739 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e4483ac44174b05872e141c17b705da with delete_type TABLET_DATA_DELETED (replaced by tablet e3db64d2695c4149887528b59b18df5f) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.827838 28653 tablet_replica.cc:330] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.827920 28653 raft_consensus.cc:2229] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.827888 27853 consensus_queue.cc:1035] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.827955 28653 raft_consensus.cc:2258] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Raft consensus is shut down!
W20240503 02:09:18.828083 27659 leader_election.cc:341] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
W20240503 02:09:18.828146 27659 leader_election.cc:341] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): Illegal state: must be running to vote when last-logged opid is not known
I20240503 02:09:18.828577 28661 ts_tablet_manager.cc:1875] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.829995 28661 ts_tablet_manager.cc:1888] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.830039 28661 log.cc:1196] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/0e4483ac44174b05872e141c17b705da
I20240503 02:09:18.830183 28661 ts_tablet_manager.cc:1909] T 0e4483ac44174b05872e141c17b705da P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.830613 27585 raft_consensus.cc:2454] T 51875ccc8e4e4ff98b74eaf7118c20c7 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.831064 27656 leader_election.cc:302] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters: bd4b79bd8b57443faa6fa5e464570188
I20240503 02:09:18.831153 27853 raft_consensus.cc:2781] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.831187 27853 raft_consensus.cc:686] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.831218 27853 consensus_queue.cc:237] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.831607 28653 ts_tablet_manager.cc:1875] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.832437 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 0e4483ac44174b05872e141c17b705da (table default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]) successfully deleted
I20240503 02:09:18.832791 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 0e4483ac44174b05872e141c17b705da (table default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]) successfully deleted
I20240503 02:09:18.832911 28653 ts_tablet_manager.cc:1888] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20240503 02:09:18.832959 28653 log.cc:1196] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/0e4483ac44174b05872e141c17b705da
I20240503 02:09:18.833101 28653 ts_tablet_manager.cc:1909] T 0e4483ac44174b05872e141c17b705da P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.834030 27198 catalog_manager.cc:5484] T 51875ccc8e4e4ff98b74eaf7118c20c7 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.834439 28766 consensus_queue.cc:1035] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.834700 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet 061caedc2c0647a4ae151e296643e779 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.835017 28664 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.835062 28664 ts_tablet_manager.cc:1367] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.000s
I20240503 02:09:18.835217 28664 raft_consensus.cc:348] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.835283 28664 raft_consensus.cc:374] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.835307 28664 raft_consensus.cc:729] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.835340 28664 consensus_queue.cc:260] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.835441 28664 ts_tablet_manager.cc:1398] T e3db64d2695c4149887528b59b18df5f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.835832 28655 tablet_bootstrap.cc:492] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.835876 28655 ts_tablet_manager.cc:1367] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20240503 02:09:18.835887 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 26d587597b0e4243ad98a7f20a1043f5 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.835919 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 0e4483ac44174b05872e141c17b705da (table default.Table_e2e3322e21f94717961d13fd7b1971ae [id=456924a45e584b8bb8dd216f131cd6e8]) successfully deleted
I20240503 02:09:18.835994 28655 raft_consensus.cc:348] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.836042 28655 raft_consensus.cc:374] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.836059 28655 raft_consensus.cc:729] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.836092 28655 consensus_queue.cc:260] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.836190 28655 ts_tablet_manager.cc:1398] T 51875ccc8e4e4ff98b74eaf7118c20c7 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.836264 28655 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.836617 28655 tablet_bootstrap.cc:654] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.837144 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 26d587597b0e4243ad98a7f20a1043f5
I20240503 02:09:18.837225 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 26d587597b0e4243ad98a7f20a1043f5 deleted and replaced by tablet 5993841aa4a9416fb3d0294d09e98d79
I20240503 02:09:18.837605 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5993841aa4a9416fb3d0294d09e98d79) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.837673 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 061caedc2c0647a4ae151e296643e779
I20240503 02:09:18.837754 27200 catalog_manager.cc:6554] ReplaceTablet: tablet 061caedc2c0647a4ae151e296643e779 deleted and replaced by tablet 0f371b6eed7c46e1bf6129a913418cf3
I20240503 02:09:18.838007 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet 061caedc2c0647a4ae151e296643e779 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f371b6eed7c46e1bf6129a913418cf3) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.838604 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5993841aa4a9416fb3d0294d09e98d79) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.838655 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 061caedc2c0647a4ae151e296643e779 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f371b6eed7c46e1bf6129a913418cf3) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.838716 28653 tablet_replica.cc:330] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.838784 28653 raft_consensus.cc:2229] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.838840 28653 raft_consensus.cc:2258] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.839390 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d587597b0e4243ad98a7f20a1043f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 5993841aa4a9416fb3d0294d09e98d79) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.839474 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 061caedc2c0647a4ae151e296643e779 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f371b6eed7c46e1bf6129a913418cf3) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.840121 28653 ts_tablet_manager.cc:1875] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.841136 27560 tablet_service.cc:1467] Processing CreateTablet for tablet 5993841aa4a9416fb3d0294d09e98d79 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.841256 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993841aa4a9416fb3d0294d09e98d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.841259 28653 ts_tablet_manager.cc:1888] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.841297 28653 log.cc:1196] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/26d587597b0e4243ad98a7f20a1043f5
I20240503 02:09:18.841480 28653 ts_tablet_manager.cc:1909] T 26d587597b0e4243ad98a7f20a1043f5 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.841836 28653 tablet_replica.cc:330] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.841909 28653 raft_consensus.cc:2229] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.841964 28653 raft_consensus.cc:2258] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.842051 28653 ts_tablet_manager.cc:1875] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.842571 27702 tablet_service.cc:1467] Processing CreateTablet for tablet 5993841aa4a9416fb3d0294d09e98d79 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.842666 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993841aa4a9416fb3d0294d09e98d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.843221 28653 ts_tablet_manager.cc:1888] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.843463 28653 log.cc:1196] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/061caedc2c0647a4ae151e296643e779
I20240503 02:09:18.843482 28693 consensus_queue.cc:1035] T 4831e4a404b34b88b0511d2e5eca0897 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.843637 28653 ts_tablet_manager.cc:1909] T 061caedc2c0647a4ae151e296643e779 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.843930 28664 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.844295 28664 tablet_bootstrap.cc:654] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.844499 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 26d587597b0e4243ad98a7f20a1043f5 (table default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]) successfully deleted
I20240503 02:09:18.844554 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 061caedc2c0647a4ae151e296643e779 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]) successfully deleted
I20240503 02:09:18.844712 27425 tablet_service.cc:1467] Processing CreateTablet for tablet 5993841aa4a9416fb3d0294d09e98d79 (DEFAULT_TABLE table=default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.844807 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5993841aa4a9416fb3d0294d09e98d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.844712 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 0f371b6eed7c46e1bf6129a913418cf3 (DEFAULT_TABLE table=default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.844910 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f371b6eed7c46e1bf6129a913418cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.845043 28656 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.845100 28656 ts_tablet_manager.cc:1367] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:18.845224 28656 raft_consensus.cc:348] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.845258 28655 tablet_bootstrap.cc:492] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.845278 28656 raft_consensus.cc:374] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.845295 28656 raft_consensus.cc:729] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.845291 28655 ts_tablet_manager.cc:1367] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240503 02:09:18.845353 28656 consensus_queue.cc:260] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.845446 28656 ts_tablet_manager.cc:1398] T e3db64d2695c4149887528b59b18df5f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.845464 28655 raft_consensus.cc:348] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.845504 28656 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.845525 28655 raft_consensus.cc:374] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.845543 28655 raft_consensus.cc:729] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.845577 28655 consensus_queue.cc:260] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.845664 28655 ts_tablet_manager.cc:1398] T e3db64d2695c4149887528b59b18df5f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.845846 28656 tablet_bootstrap.cc:654] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.846474 28655 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.846836 28655 tablet_bootstrap.cc:654] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.847498 28655 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.847517 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 0f371b6eed7c46e1bf6129a913418cf3 (DEFAULT_TABLE table=default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.847559 28655 ts_tablet_manager.cc:1367] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20240503 02:09:18.847621 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f371b6eed7c46e1bf6129a913418cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.847685 28655 raft_consensus.cc:348] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.847733 28655 raft_consensus.cc:374] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.847749 28655 raft_consensus.cc:729] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.847782 28655 consensus_queue.cc:260] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.847862 28655 ts_tablet_manager.cc:1398] T 5993841aa4a9416fb3d0294d09e98d79 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.847918 28655 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.847972 28766 raft_consensus.cc:482] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.848018 28766 raft_consensus.cc:504] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.848110 28766 leader_election.cc:288] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.848240 28655 tablet_bootstrap.cc:654] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.848490 28656 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.848527 28656 ts_tablet_manager.cc:1367] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.848603 28656 raft_consensus.cc:348] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.848645 28656 raft_consensus.cc:374] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.848661 28656 raft_consensus.cc:729] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.848695 28656 consensus_queue.cc:260] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.848773 28656 ts_tablet_manager.cc:1398] T 5993841aa4a9416fb3d0294d09e98d79 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.848832 28656 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.849043 28656 tablet_bootstrap.cc:654] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.849196 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58bc9cb8a18f4c299698671b49ec33bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.849256 27587 raft_consensus.cc:2454] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.849550 28655 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.849587 28655 ts_tablet_manager.cc:1367] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20240503 02:09:18.849632 27386 leader_election.cc:302] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.849716 28655 raft_consensus.cc:348] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.849764 28655 raft_consensus.cc:374] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.849781 28655 raft_consensus.cc:729] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.849819 28655 consensus_queue.cc:260] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.849877 28766 raft_consensus.cc:2781] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.849902 28655 ts_tablet_manager.cc:1398] T 0f371b6eed7c46e1bf6129a913418cf3 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.849912 28766 raft_consensus.cc:482] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.849931 28766 raft_consensus.cc:3037] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.850490 27692 tablet_service.cc:1467] Processing CreateTablet for tablet 0f371b6eed7c46e1bf6129a913418cf3 (DEFAULT_TABLE table=default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.850522 28766 raft_consensus.cc:504] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.850574 27692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f371b6eed7c46e1bf6129a913418cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.850620 28766 leader_election.cc:288] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.850689 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58bc9cb8a18f4c299698671b49ec33bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.850742 27722 raft_consensus.cc:2454] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.851732 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58bc9cb8a18f4c299698671b49ec33bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.851786 27587 raft_consensus.cc:3037] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.851876 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58bc9cb8a18f4c299698671b49ec33bf" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.851924 27722 raft_consensus.cc:3037] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.852389 27587 raft_consensus.cc:2454] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.852492 27722 raft_consensus.cc:2454] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.852572 27386 leader_election.cc:302] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.852883 28654 tablet_replica.cc:330] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.852974 28654 raft_consensus.cc:2229] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.853011 28654 raft_consensus.cc:2258] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.853212 28661 tablet_replica.cc:330] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.853278 28661 raft_consensus.cc:2229] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.853335 28661 raft_consensus.cc:2258] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.853451 28661 ts_tablet_manager.cc:1875] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.853894 28766 raft_consensus.cc:2781] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.853947 28766 raft_consensus.cc:686] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.853986 28766 consensus_queue.cc:237] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.854525 28654 ts_tablet_manager.cc:1875] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.855120 27200 master_service.cc:946] ReplaceTablet: received request to replace tablet ce779dcce25548eba2fc39908daf2b9a from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.855522 27197 master_service.cc:946] ReplaceTablet: received request to replace tablet 36f7c3e3e6e643948b8668c675a20b89 from {username='slave'} at 127.0.0.1:39874
I20240503 02:09:18.855561 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3a6a4426c6844434bffe8904438b4408" } modify_external_catalogs: true
I20240503 02:09:18.855625 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_3a6a4426c6844434bffe8904438b4408" } modify_external_catalogs: true
I20240503 02:09:18.856231 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 0f371b6eed7c46e1bf6129a913418cf3 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]). Expected version 1 got 0
W20240503 02:09:18.856307 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 0f371b6eed7c46e1bf6129a913418cf3 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0f371b6eed7c46e1bf6129a913418cf3 consensus state has no leader
I20240503 02:09:18.856873 27200 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet ce779dcce25548eba2fc39908daf2b9a
I20240503 02:09:18.856966 27200 catalog_manager.cc:6554] ReplaceTablet: tablet ce779dcce25548eba2fc39908daf2b9a deleted and replaced by tablet fef1b24c5c0943ee8305943dc2f1f01f
I20240503 02:09:18.857468 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet b6c9984b7488438a863dcf791b5b794f
I20240503 02:09:18.857712 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c9984b7488438a863dcf791b5b794f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.857784 28653 tablet_replica.cc:330] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.857834 28653 raft_consensus.cc:2229] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.857856 27560 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c9984b7488438a863dcf791b5b794f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.857905 28653 raft_consensus.cc:2258] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.858109 28653 ts_tablet_manager.cc:1875] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.858793 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet ce779dcce25548eba2fc39908daf2b9a with delete_type TABLET_DATA_DELETED (replaced by tablet fef1b24c5c0943ee8305943dc2f1f01f) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.859406 27426 tablet_service.cc:1467] Processing CreateTablet for tablet fef1b24c5c0943ee8305943dc2f1f01f (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.859505 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef1b24c5c0943ee8305943dc2f1f01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.859416 28653 ts_tablet_manager.cc:1888] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.860280 28653 log.cc:1196] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/b6c9984b7488438a863dcf791b5b794f
I20240503 02:09:18.860446 28656 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.860478 28653 ts_tablet_manager.cc:1909] T b6c9984b7488438a863dcf791b5b794f P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.860484 28656 ts_tablet_manager.cc:1367] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.000s
I20240503 02:09:18.860574 28656 raft_consensus.cc:348] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.860611 28656 raft_consensus.cc:374] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.860621 28656 raft_consensus.cc:729] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.860651 28653 tablet_replica.cc:330] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.860651 28656 consensus_queue.cc:260] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.860703 28653 raft_consensus.cc:2229] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.860692 27703 tablet_service.cc:1467] Processing CreateTablet for tablet fef1b24c5c0943ee8305943dc2f1f01f (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.860731 28653 raft_consensus.cc:2258] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.860733 28656 ts_tablet_manager.cc:1398] T 0f371b6eed7c46e1bf6129a913418cf3 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.860795 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet b6c9984b7488438a863dcf791b5b794f (table default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]) successfully deleted
I20240503 02:09:18.860822 28653 ts_tablet_manager.cc:1875] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.860287 28409 raft_consensus.cc:482] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.861407 28409 raft_consensus.cc:504] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.861506 28409 leader_election.cc:288] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.861558 28655 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.861912 28655 tablet_bootstrap.cc:654] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.861923 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9452139ce42b1a5dbedb19ba89046" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.861922 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9452139ce42b1a5dbedb19ba89046" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.861980 27722 raft_consensus.cc:2454] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.861982 27585 raft_consensus.cc:2454] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.862103 27386 leader_election.cc:302] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.862099 27198 catalog_manager.cc:5484] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.862181 28409 raft_consensus.cc:2781] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.862190 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.862210 28409 raft_consensus.cc:482] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.862227 28409 raft_consensus.cc:3037] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.862610 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_e6f80f24c82c449091c915a8864dbbd4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.862715 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6f80f24c82c449091c915a8864dbbd4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.862813 28409 raft_consensus.cc:504] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.862921 28409 leader_election.cc:288] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.863050 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9452139ce42b1a5dbedb19ba89046" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.863051 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9452139ce42b1a5dbedb19ba89046" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.863106 27722 raft_consensus.cc:3037] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.863106 27585 raft_consensus.cc:3037] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.863631 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 36f7c3e3e6e643948b8668c675a20b89
I20240503 02:09:18.863677 27585 raft_consensus.cc:2454] T b9c9452139ce42b1a5dbedb19ba89046 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.863677 27722 raft_consensus.cc:2454] T b9c9452139ce42b1a5dbedb19ba89046 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.863739 27197 catalog_manager.cc:6554] ReplaceTablet: tablet 36f7c3e3e6e643948b8668c675a20b89 deleted and replaced by tablet 3f8221b57363453fae2da489b651b76b
I20240503 02:09:18.863834 27386 leader_election.cc:302] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.863929 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f8221b57363453fae2da489b651b76b) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.864089 28409 raft_consensus.cc:2781] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.864094 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f8221b57363453fae2da489b651b76b) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.864135 28409 raft_consensus.cc:686] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.864179 28409 consensus_queue.cc:237] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.864291 27560 tablet_service.cc:1467] Processing CreateTablet for tablet fef1b24c5c0943ee8305943dc2f1f01f (DEFAULT_TABLE table=default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.864385 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef1b24c5c0943ee8305943dc2f1f01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.865070 28653 ts_tablet_manager.cc:1888] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.865116 28653 log.cc:1196] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/ce779dcce25548eba2fc39908daf2b9a
I20240503 02:09:18.865288 28653 ts_tablet_manager.cc:1909] T ce779dcce25548eba2fc39908daf2b9a P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.865587 28656 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.865795 28656 tablet_bootstrap.cc:654] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.866283 28655 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.866325 28655 ts_tablet_manager.cc:1367] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240503 02:09:18.866370 27702 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c9984b7488438a863dcf791b5b794f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.866452 28655 raft_consensus.cc:348] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.866498 28655 raft_consensus.cc:374] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.866513 28655 raft_consensus.cc:729] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.866552 28655 consensus_queue.cc:260] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.866636 28655 ts_tablet_manager.cc:1398] T fef1b24c5c0943ee8305943dc2f1f01f P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.867202 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet 36f7c3e3e6e643948b8668c675a20b89 with delete_type TABLET_DATA_DELETED (replaced by tablet 3f8221b57363453fae2da489b651b76b) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.867307 27692 tablet_service.cc:1514] Processing DeleteTablet for tablet ce779dcce25548eba2fc39908daf2b9a with delete_type TABLET_DATA_DELETED (replaced by tablet fef1b24c5c0943ee8305943dc2f1f01f) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.867375 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet ce779dcce25548eba2fc39908daf2b9a with delete_type TABLET_DATA_DELETED (replaced by tablet fef1b24c5c0943ee8305943dc2f1f01f) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.867848 28653 tablet_replica.cc:330] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.867918 28653 raft_consensus.cc:2229] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.867964 28653 raft_consensus.cc:2258] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.868007 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fef1b24c5c0943ee8305943dc2f1f01f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.869540 28653 ts_tablet_manager.cc:1875] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.869683 28661 ts_tablet_manager.cc:1888] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.869866 28661 log.cc:1196] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/061caedc2c0647a4ae151e296643e779
I20240503 02:09:18.870029 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet ce779dcce25548eba2fc39908daf2b9a (table default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]) successfully deleted
I20240503 02:09:18.870168 27197 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6bb6cbabe9a24968a5d1a05075cd5689" } modify_external_catalogs: true
I20240503 02:09:18.870232 27197 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_6bb6cbabe9a24968a5d1a05075cd5689" } modify_external_catalogs: true
I20240503 02:09:18.870765 28653 ts_tablet_manager.cc:1888] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.870743 27200 catalog_manager.cc:5484] T b9c9452139ce42b1a5dbedb19ba89046 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.870816 28653 log.cc:1196] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/36f7c3e3e6e643948b8668c675a20b89
I20240503 02:09:18.871001 28653 ts_tablet_manager.cc:1909] T 36f7c3e3e6e643948b8668c675a20b89 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.871069 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8221b57363453fae2da489b651b76b (DEFAULT_TABLE table=default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.871182 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8221b57363453fae2da489b651b76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.869426 28654 ts_tablet_manager.cc:1888] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.871728 28654 log.cc:1196] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/26d587597b0e4243ad98a7f20a1043f5
I20240503 02:09:18.870350 28507 consensus_queue.cc:1035] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20240503 02:09:18.870412 28656 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.871814 28656 ts_tablet_manager.cc:1367] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240503 02:09:18.871145 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8221b57363453fae2da489b651b76b (DEFAULT_TABLE table=default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.871899 28656 raft_consensus.cc:348] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.871943 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8221b57363453fae2da489b651b76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.871966 28654 ts_tablet_manager.cc:1909] T 26d587597b0e4243ad98a7f20a1043f5 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.871973 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 36f7c3e3e6e643948b8668c675a20b89 (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]) successfully deleted
I20240503 02:09:18.871942 28656 raft_consensus.cc:374] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.872046 28656 raft_consensus.cc:729] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.872082 28656 consensus_queue.cc:260] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.872205 28655 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.872242 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8221b57363453fae2da489b651b76b (DEFAULT_TABLE table=default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.872318 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8221b57363453fae2da489b651b76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.872623 28655 tablet_bootstrap.cc:654] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.872658 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 26d587597b0e4243ad98a7f20a1043f5 (table default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]) successfully deleted
I20240503 02:09:18.872162 28654 tablet_replica.cc:330] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.873304 28654 raft_consensus.cc:2229] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.873837 28654 raft_consensus.cc:2258] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.873951 28654 ts_tablet_manager.cc:1875] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.872179 28656 ts_tablet_manager.cc:1398] T fef1b24c5c0943ee8305943dc2f1f01f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.874500 27720 raft_consensus.cc:1261] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.874688 27585 raft_consensus.cc:1261] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Refusing update from remote peer bd4b79bd8b57443faa6fa5e464570188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240503 02:09:18.874809 28656 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.874881 28766 consensus_queue.cc:1035] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.874977 28766 consensus_queue.cc:1035] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.875054 27197 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet 9dc07eb64969465fb2b152337dc82851
I20240503 02:09:18.875149 28656 tablet_bootstrap.cc:654] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.875558 27703 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc07eb64969465fb2b152337dc82851 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:43176
I20240503 02:09:18.875571 27425 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc07eb64969465fb2b152337dc82851 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:34416
I20240503 02:09:18.875633 28653 tablet_replica.cc:330] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: stopping tablet replica
I20240503 02:09:18.875680 28653 raft_consensus.cc:2229] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.875716 28653 raft_consensus.cc:2258] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.875818 28653 ts_tablet_manager.cc:1875] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.875978 28661 ts_tablet_manager.cc:1909] T 061caedc2c0647a4ae151e296643e779 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.876305 28766 raft_consensus.cc:482] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.876354 28766 raft_consensus.cc:504] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.876437 28766 leader_election.cc:288] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.876607 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b786eb5fd924ef1a7ae99a7761239c9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.876677 27587 raft_consensus.cc:2454] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.876653 28661 tablet_replica.cc:330] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.876799 27386 leader_election.cc:302] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.876873 28766 raft_consensus.cc:2781] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.876902 28766 raft_consensus.cc:482] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.876924 28766 raft_consensus.cc:3037] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.876824 28661 raft_consensus.cc:2229] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.876999 28653 ts_tablet_manager.cc:1888] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.877049 28653 log.cc:1196] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-0/wal/wals/9dc07eb64969465fb2b152337dc82851
I20240503 02:09:18.877499 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b786eb5fd924ef1a7ae99a7761239c9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.877565 27718 raft_consensus.cc:2454] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 0.
I20240503 02:09:18.877673 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 061caedc2c0647a4ae151e296643e779 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]) successfully deleted
I20240503 02:09:18.877794 28653 ts_tablet_manager.cc:1909] T 9dc07eb64969465fb2b152337dc82851 P bd4b79bd8b57443faa6fa5e464570188: Deleting consensus metadata
I20240503 02:09:18.877807 28766 raft_consensus.cc:504] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.877840 27200 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.877933 28766 leader_election.cc:288] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Requested vote from peers c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.878167 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b786eb5fd924ef1a7ae99a7761239c9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.878226 27587 raft_consensus.cc:3037] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.878337 27718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b786eb5fd924ef1a7ae99a7761239c9" candidate_uuid: "bd4b79bd8b57443faa6fa5e464570188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.878391 27718 raft_consensus.cc:3037] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.878402 27557 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dc07eb64969465fb2b152337dc82851 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-03 02:09:18 UTC) from {username='slave'} at 127.0.0.1:42360
I20240503 02:09:18.878432 27190 catalog_manager.cc:4830] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597): tablet 9dc07eb64969465fb2b152337dc82851 (table default.Table_6bb6cbabe9a24968a5d1a05075cd5689 [id=dcaa66ed9577423ebe462a9302aa4a91]) successfully deleted
I20240503 02:09:18.878551 27198 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 0f371b6eed7c46e1bf6129a913418cf3 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]). Expected version 1 got 0
I20240503 02:09:18.878794 27587 raft_consensus.cc:2454] T 0b786eb5fd924ef1a7ae99a7761239c9 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.878930 27386 leader_election.cc:302] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.878998 27718 raft_consensus.cc:2454] T 0b786eb5fd924ef1a7ae99a7761239c9 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4b79bd8b57443faa6fa5e464570188 in term 1.
I20240503 02:09:18.879065 28767 raft_consensus.cc:2781] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.879112 28767 raft_consensus.cc:686] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [term 1 LEADER]: Becoming Leader. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Running, Role: LEADER
I20240503 02:09:18.879145 28767 consensus_queue.cc:237] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.879227 27557 tablet_service.cc:1467] Processing CreateTablet for tablet cc84041f67bf49e09d1bf3beda3bfc7d (DEFAULT_TABLE table=default.Table_e6f80f24c82c449091c915a8864dbbd4 [id=e034f176e262474aa82c47b8332bfa5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.879390 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc84041f67bf49e09d1bf3beda3bfc7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.879957 27197 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 58bc9cb8a18f4c299698671b49ec33bf (table default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a]). Expected version 1 got 0
I20240503 02:09:18.880259 27703 tablet_service.cc:1467] Processing CreateTablet for tablet cc84041f67bf49e09d1bf3beda3bfc7d (DEFAULT_TABLE table=default.Table_e6f80f24c82c449091c915a8864dbbd4 [id=e034f176e262474aa82c47b8332bfa5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.880344 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc84041f67bf49e09d1bf3beda3bfc7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.882054 28654 ts_tablet_manager.cc:1888] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.882009 27198 catalog_manager.cc:5484] T 0b786eb5fd924ef1a7ae99a7761239c9 P bd4b79bd8b57443faa6fa5e464570188 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1). New cstate: current_term: 1 leader_uuid: "bd4b79bd8b57443faa6fa5e464570188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } }
I20240503 02:09:18.882107 28654 log.cc:1196] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/061caedc2c0647a4ae151e296643e779
I20240503 02:09:18.882282 28654 ts_tablet_manager.cc:1909] T 061caedc2c0647a4ae151e296643e779 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.882364 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ead53984afbe4d5e95e577b38593d0e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.882529 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ead53984afbe4d5e95e577b38593d0e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.882949 29643 raft_consensus.cc:482] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.882998 29643 raft_consensus.cc:504] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.883116 29643 leader_election.cc:288] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.883126 28664 tablet_bootstrap.cc:492] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.883169 28664 ts_tablet_manager.cc:1367] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.000s
I20240503 02:09:18.883311 28664 raft_consensus.cc:348] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.883373 28664 raft_consensus.cc:374] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.883389 28664 raft_consensus.cc:729] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.883396 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fd56a81cf4467c937e670ee2703c47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.883424 28664 consensus_queue.cc:260] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.883456 27447 raft_consensus.cc:2454] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.883442 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fd56a81cf4467c937e670ee2703c47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.883492 27587 raft_consensus.cc:2454] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.883519 28664 ts_tablet_manager.cc:1398] T 5993841aa4a9416fb3d0294d09e98d79 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.883589 28664 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.883827 27659 leader_election.cc:302] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.883908 28774 raft_consensus.cc:2781] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.883939 28774 raft_consensus.cc:482] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.883955 28774 raft_consensus.cc:3037] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.884024 28664 tablet_bootstrap.cc:654] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.884538 28774 raft_consensus.cc:504] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.884590 28664 tablet_bootstrap.cc:492] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.884621 28664 ts_tablet_manager.cc:1367] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.884635 28774 leader_election.cc:288] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.884760 28664 raft_consensus.cc:348] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.884810 28664 raft_consensus.cc:374] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.884825 28664 raft_consensus.cc:729] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.884857 28664 consensus_queue.cc:260] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.884951 28664 ts_tablet_manager.cc:1398] T 0f371b6eed7c46e1bf6129a913418cf3 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.885000 28664 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.885205 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fd56a81cf4467c937e670ee2703c47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.885268 27447 raft_consensus.cc:3037] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.885335 28664 tablet_bootstrap.cc:654] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.885741 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56fd56a81cf4467c937e670ee2703c47" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.885795 27587 raft_consensus.cc:3037] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.885885 27447 raft_consensus.cc:2454] T 56fd56a81cf4467c937e670ee2703c47 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.886036 27659 leader_election.cc:302] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.886127 28774 raft_consensus.cc:2781] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.886169 28774 raft_consensus.cc:686] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.886204 28774 consensus_queue.cc:237] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.886348 27587 raft_consensus.cc:2454] T 56fd56a81cf4467c937e670ee2703c47 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.886611 27426 tablet_service.cc:1467] Processing CreateTablet for tablet cc84041f67bf49e09d1bf3beda3bfc7d (DEFAULT_TABLE table=default.Table_e6f80f24c82c449091c915a8864dbbd4 [id=e034f176e262474aa82c47b8332bfa5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.886765 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc84041f67bf49e09d1bf3beda3bfc7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.887327 27198 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 0f371b6eed7c46e1bf6129a913418cf3 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]). Expected version 1 got 0
I20240503 02:09:18.887375 27198 catalog_manager.cc:5484] T 56fd56a81cf4467c937e670ee2703c47 P c2be543b8aad4e80a58d6b4ab669faae reported cstate change: term changed from 0 to 1, leader changed from <none> to c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3). New cstate: current_term: 1 leader_uuid: "c2be543b8aad4e80a58d6b4ab669faae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } health_report { overall_health: HEALTHY } } }
I20240503 02:09:18.888690 28654 tablet_replica.cc:330] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.888748 28654 raft_consensus.cc:2229] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.888743 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 061caedc2c0647a4ae151e296643e779 (table default.Table_38fa31d9bcc54033a6c87470b7d1de63 [id=1fde4a81298d48789d1266e6bb3fac80]) successfully deleted
I20240503 02:09:18.888813 28654 raft_consensus.cc:2258] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.877013 28661 raft_consensus.cc:2258] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.889056 28661 ts_tablet_manager.cc:1875] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.889143 28655 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.889182 28655 ts_tablet_manager.cc:1367] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240503 02:09:18.889297 28655 raft_consensus.cc:348] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.889356 28655 raft_consensus.cc:374] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.889374 28655 raft_consensus.cc:729] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.889622 28655 consensus_queue.cc:260] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.889720 28655 ts_tablet_manager.cc:1398] T 3f8221b57363453fae2da489b651b76b P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.889786 28655 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.890118 28655 tablet_bootstrap.cc:654] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.890537 28661 ts_tablet_manager.cc:1888] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.890586 28661 log.cc:1196] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/26d587597b0e4243ad98a7f20a1043f5
I20240503 02:09:18.890760 28661 ts_tablet_manager.cc:1909] T 26d587597b0e4243ad98a7f20a1043f5 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.890947 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 431b0d0ada1f42ce9519939ebbbcae92 (DEFAULT_TABLE table=default.Table_ead53984afbe4d5e95e577b38593d0e9 [id=5fb324b682c341f68d7ef2c40411ed21]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.890974 28661 tablet_replica.cc:330] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.891036 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 431b0d0ada1f42ce9519939ebbbcae92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.891068 27198 catalog_manager.cc:5576] TS bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597) does not have the latest schema for tablet 3f8221b57363453fae2da489b651b76b (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]). Expected version 1 got 0
I20240503 02:09:18.891081 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 431b0d0ada1f42ce9519939ebbbcae92 (DEFAULT_TABLE table=default.Table_ead53984afbe4d5e95e577b38593d0e9 [id=5fb324b682c341f68d7ef2c40411ed21]), partition=RANGE (key) PARTITION UNBOUNDED
W20240503 02:09:18.891134 27198 catalog_manager.cc:5676] Failed to send AlterTable RPC for tablet 3f8221b57363453fae2da489b651b76b (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3f8221b57363453fae2da489b651b76b consensus state has no leader
I20240503 02:09:18.891160 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 431b0d0ada1f42ce9519939ebbbcae92. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.891978 27198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_aa31c1a95cac47b5858d2108de3a4bd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.892061 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 26d587597b0e4243ad98a7f20a1043f5 (table default.Table_5c7eed47a0be43d4a4cd133971c8cec3 [id=50fc90fb61df47fa9f7dcb5e32813b29]) successfully deleted
I20240503 02:09:18.892320 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 431b0d0ada1f42ce9519939ebbbcae92 (DEFAULT_TABLE table=default.Table_ead53984afbe4d5e95e577b38593d0e9 [id=5fb324b682c341f68d7ef2c40411ed21]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.892424 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 431b0d0ada1f42ce9519939ebbbcae92. 1 dirs total, 0 dirs full, 0 dirs failed
W20240503 02:09:18.892079 27198 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa31c1a95cac47b5858d2108de3a4bd1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.893479 28507 consensus_queue.cc:1035] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.891039 28661 raft_consensus.cc:2229] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.893693 28661 raft_consensus.cc:2258] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.893882 28661 ts_tablet_manager.cc:1875] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.895422 28661 ts_tablet_manager.cc:1888] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.895841 29240 consensus_queue.cc:1035] T 6062b6fee7974681b5fddbea0eb1ecfc P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.897002 28661 log.cc:1196] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/36f7c3e3e6e643948b8668c675a20b89
I20240503 02:09:18.897249 28661 ts_tablet_manager.cc:1909] T 36f7c3e3e6e643948b8668c675a20b89 P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.897789 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet 36f7c3e3e6e643948b8668c675a20b89 (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]) successfully deleted
I20240503 02:09:18.898135 28661 tablet_replica.cc:330] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.898520 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 9e118b327f6b461d850b98e8cc267618 (DEFAULT_TABLE table=default.Table_aa31c1a95cac47b5858d2108de3a4bd1 [id=7510c541f2804df1805379a05327d69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.898685 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 9e118b327f6b461d850b98e8cc267618 (DEFAULT_TABLE table=default.Table_aa31c1a95cac47b5858d2108de3a4bd1 [id=7510c541f2804df1805379a05327d69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.898782 28661 raft_consensus.cc:2229] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.898919 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 9e118b327f6b461d850b98e8cc267618 (DEFAULT_TABLE table=default.Table_aa31c1a95cac47b5858d2108de3a4bd1 [id=7510c541f2804df1805379a05327d69e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.898998 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e118b327f6b461d850b98e8cc267618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.898815 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e118b327f6b461d850b98e8cc267618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.900492 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e118b327f6b461d850b98e8cc267618. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.901536 28654 ts_tablet_manager.cc:1875] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.902359 28710 tablet.cc:1722] T 58bc9cb8a18f4c299698671b49ec33bf P bd4b79bd8b57443faa6fa5e464570188: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.903596 30009 tablet.cc:1722] T 58bc9cb8a18f4c299698671b49ec33bf P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.903637 28656 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.903677 28656 ts_tablet_manager.cc:1367] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240503 02:09:18.903820 28656 raft_consensus.cc:348] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.903874 28656 raft_consensus.cc:374] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.903900 28656 raft_consensus.cc:729] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.903903 28654 ts_tablet_manager.cc:1888] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.903962 28654 log.cc:1196] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/b6c9984b7488438a863dcf791b5b794f
I20240503 02:09:18.903946 28656 consensus_queue.cc:260] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.904038 28656 ts_tablet_manager.cc:1398] T 3f8221b57363453fae2da489b651b76b P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.904048 30370 tablet.cc:1722] T 58bc9cb8a18f4c299698671b49ec33bf P c2be543b8aad4e80a58d6b4ab669faae: Alter schema from (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
0:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240503 02:09:18.904085 28656 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.904136 28654 ts_tablet_manager.cc:1909] T b6c9984b7488438a863dcf791b5b794f P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.904428 28656 tablet_bootstrap.cc:654] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.904742 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet b6c9984b7488438a863dcf791b5b794f (table default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]) successfully deleted
I20240503 02:09:18.904819 28654 tablet_replica.cc:330] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.904872 28654 raft_consensus.cc:2229] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.904920 28654 raft_consensus.cc:2258] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.904979 28661 raft_consensus.cc:2258] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.905268 28661 ts_tablet_manager.cc:1875] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.905436 28654 ts_tablet_manager.cc:1875] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.905625 27197 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: default.Table_6c53d6cacadf4be1b38892cb44c74321 [id=0a78bf10c6f24929a3816cc208e3461a] alter complete (version 1)
I20240503 02:09:18.906795 28774 consensus_queue.cc:1035] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.907307 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9d73f8ee1c2649b190826bc673e1e36b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.907408 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d73f8ee1c2649b190826bc673e1e36b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.907639 28654 ts_tablet_manager.cc:1888] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.907678 28654 log.cc:1196] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/36f7c3e3e6e643948b8668c675a20b89
I20240503 02:09:18.907963 27200 catalog_manager.cc:5576] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407) does not have the latest schema for tablet 3f8221b57363453fae2da489b651b76b (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]). Expected version 1 got 0
I20240503 02:09:18.908257 28654 ts_tablet_manager.cc:1909] T 36f7c3e3e6e643948b8668c675a20b89 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.909190 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 36f7c3e3e6e643948b8668c675a20b89 (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]) successfully deleted
I20240503 02:09:18.909291 28654 tablet_replica.cc:330] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.909430 28654 raft_consensus.cc:2229] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.909556 28654 raft_consensus.cc:2258] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.909673 28654 ts_tablet_manager.cc:1875] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.910701 28774 consensus_queue.cc:1035] T 106f9b05066d46ec83ce7e2129840eee P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.911921 28654 ts_tablet_manager.cc:1888] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.912603 28654 log.cc:1196] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/ce779dcce25548eba2fc39908daf2b9a
I20240503 02:09:18.913214 28654 ts_tablet_manager.cc:1909] T ce779dcce25548eba2fc39908daf2b9a P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.913631 27426 tablet_service.cc:1467] Processing CreateTablet for tablet bcde4d70a80b40119663204c136ad728 (DEFAULT_TABLE table=default.Table_9d73f8ee1c2649b190826bc673e1e36b [id=7652354c92e84ae1a4f241375c4af895]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.913755 27557 tablet_service.cc:1467] Processing CreateTablet for tablet bcde4d70a80b40119663204c136ad728 (DEFAULT_TABLE table=default.Table_9d73f8ee1c2649b190826bc673e1e36b [id=7652354c92e84ae1a4f241375c4af895]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.913776 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcde4d70a80b40119663204c136ad728. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.913834 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcde4d70a80b40119663204c136ad728. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.914885 27703 tablet_service.cc:1467] Processing CreateTablet for tablet bcde4d70a80b40119663204c136ad728 (DEFAULT_TABLE table=default.Table_9d73f8ee1c2649b190826bc673e1e36b [id=7652354c92e84ae1a4f241375c4af895]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.914973 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcde4d70a80b40119663204c136ad728. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.915093 28507 consensus_queue.cc:1035] T 97721d6cf071459baaa04b58a36c7013 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.916376 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_1d887222a28f47d1b2a364c1a1cae360"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20240503 02:09:18.916492 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d887222a28f47d1b2a364c1a1cae360 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.917511 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet ce779dcce25548eba2fc39908daf2b9a (table default.Table_f645395dbcd54876beef5a179ddf4814 [id=91298625fe1d4d9ba169c12e44c5209d]) successfully deleted
I20240503 02:09:18.917630 28654 tablet_replica.cc:330] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: stopping tablet replica
I20240503 02:09:18.917726 28654 raft_consensus.cc:2229] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Raft consensus shutting down.
I20240503 02:09:18.917838 28654 raft_consensus.cc:2258] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.917977 28654 ts_tablet_manager.cc:1875] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.919384 28654 ts_tablet_manager.cc:1888] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20240503 02:09:18.919566 28654 log.cc:1196] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-1/wal/wals/9dc07eb64969465fb2b152337dc82851
I20240503 02:09:18.919819 28654 ts_tablet_manager.cc:1909] T 9dc07eb64969465fb2b152337dc82851 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Deleting consensus metadata
I20240503 02:09:18.920228 27187 catalog_manager.cc:4830] TS 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407): tablet 9dc07eb64969465fb2b152337dc82851 (table default.Table_6bb6cbabe9a24968a5d1a05075cd5689 [id=dcaa66ed9577423ebe462a9302aa4a91]) successfully deleted
I20240503 02:09:18.920325 27703 tablet_service.cc:1467] Processing CreateTablet for tablet b0c5bb5c289541db93fa9bafa0e21167 (DEFAULT_TABLE table=default.Table_1d887222a28f47d1b2a364c1a1cae360 [id=25fa6a5b8eee4678be2773eee9cbd266]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.920418 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c5bb5c289541db93fa9bafa0e21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.920284 27557 tablet_service.cc:1467] Processing CreateTablet for tablet b0c5bb5c289541db93fa9bafa0e21167 (DEFAULT_TABLE table=default.Table_1d887222a28f47d1b2a364c1a1cae360 [id=25fa6a5b8eee4678be2773eee9cbd266]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.920680 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c5bb5c289541db93fa9bafa0e21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.921586 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b0c5bb5c289541db93fa9bafa0e21167 (DEFAULT_TABLE table=default.Table_1d887222a28f47d1b2a364c1a1cae360 [id=25fa6a5b8eee4678be2773eee9cbd266]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.921712 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c5bb5c289541db93fa9bafa0e21167. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.922708 28661 ts_tablet_manager.cc:1888] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.922761 28661 log.cc:1196] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/b6c9984b7488438a863dcf791b5b794f
I20240503 02:09:18.922955 28661 ts_tablet_manager.cc:1909] T b6c9984b7488438a863dcf791b5b794f P c2be543b8aad4e80a58d6b4ab669faae: Deleting consensus metadata
I20240503 02:09:18.923229 28661 tablet_replica.cc:330] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: stopping tablet replica
I20240503 02:09:18.923352 27200 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_a83d840940b94191a760b0ab6448a7e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.923339 28664 tablet_bootstrap.cc:492] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
W20240503 02:09:18.923465 27200 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a83d840940b94191a760b0ab6448a7e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.923468 28664 ts_tablet_manager.cc:1367] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20240503 02:09:18.923478 28661 raft_consensus.cc:2229] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240503 02:09:18.923532 28661 raft_consensus.cc:2258] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240503 02:09:18.923393 27189 catalog_manager.cc:4830] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961): tablet b6c9984b7488438a863dcf791b5b794f (table default.Table_3a6a4426c6844434bffe8904438b4408 [id=a311cc5ddb714c30a708aee47e50489f]) successfully deleted
I20240503 02:09:18.923625 28664 raft_consensus.cc:348] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.923676 28664 raft_consensus.cc:374] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.923693 28664 raft_consensus.cc:729] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.923729 28664 consensus_queue.cc:260] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.923820 28664 ts_tablet_manager.cc:1398] T fef1b24c5c0943ee8305943dc2f1f01f P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.923832 28661 ts_tablet_manager.cc:1875] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:09:18.923884 28664 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.924026 27198 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_ca2d781a10ed43a999c3abfa17fc4f91" } new_table_name: "default.Table_6a05ef21fc58416eb77fe3c8a1fa5b83" modify_external_catalogs: true
I20240503 02:09:18.924237 28664 tablet_bootstrap.cc:654] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.924707 28655 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.924755 28655 ts_tablet_manager.cc:1367] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20240503 02:09:18.924896 28655 raft_consensus.cc:348] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.924957 28655 raft_consensus.cc:374] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.924973 28655 raft_consensus.cc:729] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.925010 28655 consensus_queue.cc:260] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.925112 28655 ts_tablet_manager.cc:1398] T cc84041f67bf49e09d1bf3beda3bfc7d P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.925184 28655 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.925820 27197 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_9bdfd7afd00740d580333f2cf57a284a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.925925 28655 tablet_bootstrap.cc:654] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
W20240503 02:09:18.925932 27197 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9bdfd7afd00740d580333f2cf57a284a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.926452 28656 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.926505 28656 ts_tablet_manager.cc:1367] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240503 02:09:18.926641 28656 raft_consensus.cc:348] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.926694 28656 raft_consensus.cc:374] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.926710 28656 raft_consensus.cc:729] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.926760 28656 consensus_queue.cc:260] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.926854 28656 ts_tablet_manager.cc:1398] T cc84041f67bf49e09d1bf3beda3bfc7d P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.926927 28656 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.927284 28656 tablet_bootstrap.cc:654] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.928694 28664 tablet_bootstrap.cc:492] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.928738 28664 ts_tablet_manager.cc:1367] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.928961 28664 raft_consensus.cc:348] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.929023 28664 raft_consensus.cc:374] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.929039 28664 raft_consensus.cc:729] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.929072 28664 consensus_queue.cc:260] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } }
I20240503 02:09:18.929152 28664 ts_tablet_manager.cc:1398] T 3f8221b57363453fae2da489b651b76b P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.929205 28664 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.929803 28664 tablet_bootstrap.cc:654] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.930099 27703 tablet_service.cc:1467] Processing CreateTablet for tablet 4affa644343845a7943339244912d41f (DEFAULT_TABLE table=default.Table_a83d840940b94191a760b0ab6448a7e6 [id=1625bf34c5c0449981b009b1542895bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.930164 28774 consensus_queue.cc:1035] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.930294 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4affa644343845a7943339244912d41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.930387 28655 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.930437 28655 ts_tablet_manager.cc:1367] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240503 02:09:18.930565 28655 raft_consensus.cc:348] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.930626 28655 raft_consensus.cc:374] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.930644 28655 raft_consensus.cc:729] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.930678 28655 consensus_queue.cc:260] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.930676 27196 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_af4edb9428ef444ea0557d7f7563b237"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.930755 28655 ts_tablet_manager.cc:1398] T 431b0d0ada1f42ce9519939ebbbcae92 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:18.930770 27196 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af4edb9428ef444ea0557d7f7563b237 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.930821 28655 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.931241 28655 tablet_bootstrap.cc:654] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.931819 27426 tablet_service.cc:1467] Processing CreateTablet for tablet 4affa644343845a7943339244912d41f (DEFAULT_TABLE table=default.Table_a83d840940b94191a760b0ab6448a7e6 [id=1625bf34c5c0449981b009b1542895bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.931903 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4affa644343845a7943339244912d41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.932400 28774 raft_consensus.cc:482] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.932446 28774 raft_consensus.cc:504] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.932557 28774 leader_election.cc:288] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.932723 27703 tablet_service.cc:1467] Processing CreateTablet for tablet fbf245bb651743f295dc4eaa4a83c57f (DEFAULT_TABLE table=default.Table_9bdfd7afd00740d580333f2cf57a284a [id=995493c26ef7478498d7dd377961bfc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.932804 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf245bb651743f295dc4eaa4a83c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.932793 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790208954e6433490eff4bf38b49835" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" is_pre_election: true
I20240503 02:09:18.932868 27587 raft_consensus.cc:2454] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.932888 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790208954e6433490eff4bf38b49835" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.932961 27450 raft_consensus.cc:2454] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 0.
I20240503 02:09:18.933003 27656 leader_election.cc:302] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.933109 28774 raft_consensus.cc:2781] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.933099 27557 tablet_service.cc:1467] Processing CreateTablet for tablet 4affa644343845a7943339244912d41f (DEFAULT_TABLE table=default.Table_a83d840940b94191a760b0ab6448a7e6 [id=1625bf34c5c0449981b009b1542895bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.933136 28774 raft_consensus.cc:482] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.933156 28774 raft_consensus.cc:3037] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.933183 27557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4affa644343845a7943339244912d41f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.933723 28656 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.933768 28656 ts_tablet_manager.cc:1367] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20240503 02:09:18.933802 28774 raft_consensus.cc:504] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.933890 28774 leader_election.cc:288] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), 5cc801b4e0b047b3b4b30f8dd5c7c5db (127.26.117.2:38407)
I20240503 02:09:18.933884 28656 raft_consensus.cc:348] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.933934 28656 raft_consensus.cc:374] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.933951 28656 raft_consensus.cc:729] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.933990 28656 consensus_queue.cc:260] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.934084 28656 ts_tablet_manager.cc:1398] T 431b0d0ada1f42ce9519939ebbbcae92 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.934134 28656 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.934257 27560 tablet_service.cc:1467] Processing CreateTablet for tablet fbf245bb651743f295dc4eaa4a83c57f (DEFAULT_TABLE table=default.Table_9bdfd7afd00740d580333f2cf57a284a [id=995493c26ef7478498d7dd377961bfc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.934346 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf245bb651743f295dc4eaa4a83c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.934453 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790208954e6433490eff4bf38b49835" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.934489 28656 tablet_bootstrap.cc:654] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.934511 27447 raft_consensus.cc:3037] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.935137 27447 raft_consensus.cc:2454] T e790208954e6433490eff4bf38b49835 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.935248 28655 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.935279 28655 ts_tablet_manager.cc:1367] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240503 02:09:18.935284 27659 leader_election.cc:302] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd4b79bd8b57443faa6fa5e464570188, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.935410 28655 raft_consensus.cc:348] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.935444 28661 ts_tablet_manager.cc:1888] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20240503 02:09:18.935462 28655 raft_consensus.cc:374] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.935483 28661 log.cc:1196] T ce779dcce25548eba2fc39908daf2b9a P c2be543b8aad4e80a58d6b4ab669faae: Deleting WAL directory at /tmp/dist-test-taskpsd1k2/test-tmp/master-stress-test.1.CatalogManagerConfigurations_MasterStressTest.Test_1.1714702145938236-27092-0/minicluster-data/ts-2/wal/wals/ce779dcce25548eba2fc39908daf2b9a
I20240503 02:09:18.935479 28655 raft_consensus.cc:729] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.935544 28655 consensus_queue.cc:260] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.935618 29643 raft_consensus.cc:2781] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.935629 28655 ts_tablet_manager.cc:1398] T 9e118b327f6b461d850b98e8cc267618 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.935664 29643 raft_consensus.cc:686] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [term 1 LEADER]: Becoming Leader. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Running, Role: LEADER
I20240503 02:09:18.935676 28655 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.935707 29643 consensus_queue.cc:237] T e790208954e6433490eff4bf38b49835 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.935904 27587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e790208954e6433490eff4bf38b49835" candidate_uuid: "c2be543b8aad4e80a58d6b4ab669faae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"
I20240503 02:09:18.935956 27587 raft_consensus.cc:3037] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.936031 28655 tablet_bootstrap.cc:654] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.936410 28664 tablet_bootstrap.cc:492] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.936450 28664 ts_tablet_manager.cc:1367] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240503 02:09:18.936571 27587 raft_consensus.cc:2454] T e790208954e6433490eff4bf38b49835 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2be543b8aad4e80a58d6b4ab669faae in term 1.
I20240503 02:09:18.936584 28664 raft_consensus.cc:348] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.936630 28664 raft_consensus.cc:374] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.936645 28664 raft_consensus.cc:729] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.936663 28655 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188: No bootstrap required, opened a new log
I20240503 02:09:18.936669 28664 consensus_queue.cc:260] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.936698 28655 ts_tablet_manager.cc:1367] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.936760 28664 ts_tablet_manager.cc:1398] T cc84041f67bf49e09d1bf3beda3bfc7d P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.936807 28664 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.936823 28655 raft_consensus.cc:348] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.936877 28655 raft_consensus.cc:374] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.936892 28655 raft_consensus.cc:729] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4b79bd8b57443faa6fa5e464570188, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.936928 28655 consensus_queue.cc:260] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.936980 27195 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39874:
name: "default.Table_ca77221408474cf4896164dd3e0d630c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:09:18.937045 28655 ts_tablet_manager.cc:1398] T bcde4d70a80b40119663204c136ad728 P bd4b79bd8b57443faa6fa5e464570188: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20240503 02:09:18.937070 27195 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca77221408474cf4896164dd3e0d630c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240503 02:09:18.937112 28655 tablet_bootstrap.cc:492] T b0c5bb5c289541db93fa9bafa0e21167 P bd4b79bd8b57443faa6fa5e464570188: Bootstrap starting.
I20240503 02:09:18.937167 28664 tablet_bootstrap.cc:654] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.937464 28655 tablet_bootstrap.cc:654] T b0c5bb5c289541db93fa9bafa0e21167 P bd4b79bd8b57443faa6fa5e464570188: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.937881 29241 raft_consensus.cc:482] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240503 02:09:18.937928 29241 raft_consensus.cc:504] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.938020 29241 leader_election.cc:288] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.938194 27450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5b46cbf4bc4b37b463616d0f8c5108" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188" is_pre_election: true
I20240503 02:09:18.938207 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5b46cbf4bc4b37b463616d0f8c5108" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae" is_pre_election: true
I20240503 02:09:18.938256 27450 raft_consensus.cc:2454] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.938268 27722 raft_consensus.cc:2454] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 0.
I20240503 02:09:18.938380 27523 leader_election.cc:302] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, c2be543b8aad4e80a58d6b4ab669faae; no voters:
I20240503 02:09:18.938470 29241 raft_consensus.cc:2781] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240503 02:09:18.938521 29241 raft_consensus.cc:482] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240503 02:09:18.938545 29241 raft_consensus.cc:3037] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.938549 28774 consensus_queue.cc:1035] T 941c3021996a4962b87b068018948e67 P c2be543b8aad4e80a58d6b4ab669faae [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.938812 27560 tablet_service.cc:1467] Processing CreateTablet for tablet a84b93f9b2894913b89adddd7c38dcd9 (DEFAULT_TABLE table=default.Table_af4edb9428ef444ea0557d7f7563b237 [id=1707baaf546d4994a1bebe327632c33d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.938875 28656 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.938911 27560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84b93f9b2894913b89adddd7c38dcd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.938916 28656 ts_tablet_manager.cc:1367] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20240503 02:09:18.939137 28656 raft_consensus.cc:348] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.939188 28656 raft_consensus.cc:374] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.939172 29241 raft_consensus.cc:504] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.939281 29241 leader_election.cc:288] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Requested vote from peers bd4b79bd8b57443faa6fa5e464570188 (127.26.117.1:46597), c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961)
I20240503 02:09:18.939409 27447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5b46cbf4bc4b37b463616d0f8c5108" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd4b79bd8b57443faa6fa5e464570188"
I20240503 02:09:18.939462 27447 raft_consensus.cc:3037] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.939208 28656 raft_consensus.cc:729] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.939615 28656 consensus_queue.cc:260] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.939716 28656 ts_tablet_manager.cc:1398] T 9e118b327f6b461d850b98e8cc267618 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.939764 28656 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.939791 28664 tablet_bootstrap.cc:492] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.939829 28664 ts_tablet_manager.cc:1367] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.939944 28664 raft_consensus.cc:348] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.939992 28664 raft_consensus.cc:374] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.940009 28664 raft_consensus.cc:729] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.940047 28664 consensus_queue.cc:260] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.940068 27447 raft_consensus.cc:2454] T 5d5b46cbf4bc4b37b463616d0f8c5108 P bd4b79bd8b57443faa6fa5e464570188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.940133 28664 ts_tablet_manager.cc:1398] T 431b0d0ada1f42ce9519939ebbbcae92 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.940184 28664 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.940168 28656 tablet_bootstrap.cc:654] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.940305 27722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d5b46cbf4bc4b37b463616d0f8c5108" candidate_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be543b8aad4e80a58d6b4ab669faae"
I20240503 02:09:18.940383 27722 raft_consensus.cc:3037] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:09:18.940541 28664 tablet_bootstrap.cc:654] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.940779 28656 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.940819 28656 ts_tablet_manager.cc:1367] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240503 02:09:18.940896 27524 leader_election.cc:302] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc801b4e0b047b3b4b30f8dd5c7c5db, bd4b79bd8b57443faa6fa5e464570188; no voters:
I20240503 02:09:18.940943 28656 raft_consensus.cc:348] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.940991 28656 raft_consensus.cc:374] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.941008 28656 raft_consensus.cc:729] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.941043 28656 consensus_queue.cc:260] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.941066 27722 raft_consensus.cc:2454] T 5d5b46cbf4bc4b37b463616d0f8c5108 P c2be543b8aad4e80a58d6b4ab669faae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc801b4e0b047b3b4b30f8dd5c7c5db in term 1.
I20240503 02:09:18.941107 29241 raft_consensus.cc:2781] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:09:18.941151 29241 raft_consensus.cc:686] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 1 LEADER]: Becoming Leader. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Running, Role: LEADER
I20240503 02:09:18.941177 28767 consensus_queue.cc:1035] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.941197 29241 consensus_queue.cc:237] T 5d5b46cbf4bc4b37b463616d0f8c5108 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.941545 27195 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9cb244f396f642349217ee1381639223" } modify_external_catalogs: true
I20240503 02:09:18.941623 27195 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_9cb244f396f642349217ee1381639223" } modify_external_catalogs: true
I20240503 02:09:18.941660 28664 tablet_bootstrap.cc:492] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.941699 28664 ts_tablet_manager.cc:1367] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240503 02:09:18.941154 28656 ts_tablet_manager.cc:1398] T bcde4d70a80b40119663204c136ad728 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.941845 28656 tablet_bootstrap.cc:492] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Bootstrap starting.
I20240503 02:09:18.941823 28664 raft_consensus.cc:348] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.941890 28664 raft_consensus.cc:374] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.941908 28664 raft_consensus.cc:729] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be543b8aad4e80a58d6b4ab669faae, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.941936 28664 consensus_queue.cc:260] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.942023 28664 ts_tablet_manager.cc:1398] T 9e118b327f6b461d850b98e8cc267618 P c2be543b8aad4e80a58d6b4ab669faae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.942072 28664 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P c2be543b8aad4e80a58d6b4ab669faae: Bootstrap starting.
I20240503 02:09:18.942224 28656 tablet_bootstrap.cc:654] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.942207 27703 tablet_service.cc:1467] Processing CreateTablet for tablet a84b93f9b2894913b89adddd7c38dcd9 (DEFAULT_TABLE table=default.Table_af4edb9428ef444ea0557d7f7563b237 [id=1707baaf546d4994a1bebe327632c33d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.942333 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84b93f9b2894913b89adddd7c38dcd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.942384 28664 tablet_bootstrap.cc:654] T bcde4d70a80b40119663204c136ad728 P c2be543b8aad4e80a58d6b4ab669faae: Neither blocks nor log segments found. Creating new log.
I20240503 02:09:18.942762 28656 tablet_bootstrap.cc:492] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: No bootstrap required, opened a new log
I20240503 02:09:18.942804 28656 ts_tablet_manager.cc:1367] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20240503 02:09:18.942770 27196 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39874:
table { table_name: "default.Table_8121b25900c6441c9ccd1c09f0a94276" } new_table_name: "default.Table_2bdbe01bdd8c49629a89e01c51a75a4d" modify_external_catalogs: true
I20240503 02:09:18.942940 28656 raft_consensus.cc:348] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.942992 28656 raft_consensus.cc:374] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:09:18.943009 28656 raft_consensus.cc:729] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc801b4e0b047b3b4b30f8dd5c7c5db, State: Initialized, Role: FOLLOWER
I20240503 02:09:18.943034 28656 consensus_queue.cc:260] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db" member_type: VOTER last_known_addr { host: "127.26.117.2" port: 38407 } }
I20240503 02:09:18.943114 28656 ts_tablet_manager.cc:1398] T b0c5bb5c289541db93fa9bafa0e21167 P 5cc801b4e0b047b3b4b30f8dd5c7c5db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20240503 02:09:18.943290 27426 tablet_service.cc:1467] Processing CreateTablet for tablet fbf245bb651743f295dc4eaa4a83c57f (DEFAULT_TABLE table=default.Table_9bdfd7afd00740d580333f2cf57a284a [id=995493c26ef7478498d7dd377961bfc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.943369 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbf245bb651743f295dc4eaa4a83c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.943768 27195 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 9e12d823f5eb49f38d5fdc7ee2185bb4: Sending DeleteTablet for 3 replicas of tablet e0b2d6d7518f44ba855a43086f24de4d
I20240503 02:09:18.944469 27425 tablet_service.cc:1467] Processing CreateTablet for tablet a84b93f9b2894913b89adddd7c38dcd9 (DEFAULT_TABLE table=default.Table_af4edb9428ef444ea0557d7f7563b237 [id=1707baaf546d4994a1bebe327632c33d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:09:18.944561 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a84b93f9b2894913b89adddd7c38dcd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:09:18.944758 27197 catalog_manager.cc:5576] TS c2be543b8aad4e80a58d6b4ab669faae (127.26.117.3:35961) does not have the latest schema for tablet 3f8221b57363453fae2da489b651b76b (table default.Table_9bf7a8d1fabf40afbe59b4c9e9de9a6e [id=cab5446f912f473cb92ec35b7dd2105f]). Expected version 1 got 0
I20240503 02:09:18.945135 28664 tablet_bootstrap.cc:492] T bcde4d70a80b40119663204c136ad728 P c2be543b8aad4e80a58d6b4ab669faae: No bootstrap required, opened a new log
I20240503 02:09:18.945173 28664 ts_tablet_manager.cc:1367] T bcde4d70a80b40119663204c136ad728 P c2be543b8aad4e80a58d6b4ab669faae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240503 02:09:18.945147 28685 consensus_queue.cc:1035] T 7ec2373bab15403cbcae47fd20f3c7ce P bd4b79bd8b57443faa6fa5e464570188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240503 02:09:18.945295 28664 raft_consensus.cc:348] T bcde4d70a80b40119663204c136ad728 P c2be543b8aad4e80a58d6b4ab669faae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd4b79bd8b57443faa6fa5e464570188" member_type: VOTER last_known_addr { host: "127.26.117.1" port: 46597 } } peers { permanent_uuid: "c2be543b8aad4e80a58d6b4ab669faae" member_type: VOTER last_known_addr { host: "127.26.117.3" port: 35961 } } peers { permanent_uuid: "5cc801b4e0b047b3b4b30f8dd5c7c5db"